AWS AppSync with Firebase Authentication

I would like to use Firebase Auth as the Authorization service for AWS AppSync. In the AppSync settings, I tried to add Firebase Auth as an OpenID Connect provider, but it seems like I'm doing something wrong since I'm getting this error while trying to make a request with a Firebase Auth tokenId:

{
  "errors": [
    {
      "errorType": "UnauthorizedException",
      "message": "Missing authorization header"
    }
  ]
}

In AppSync im using these settings:

• Authorization type: OpenID Connect
• OpenID Connect provider domain (Issuer URL): https://securetoken.google.com/<Firebase Project ID>
• Client ID: <Firebase Api Key>
• Issued At TTL: 0
• Auth Time TTL: 0

When performing the query I use the TokenId provided by firebase, and not the ones that come from the federated identity provider(Facebook or Google for example).