Invalid OAuth response from oidc.<provider_name>

I am trying to add Azure B2C as on OIDC provider for Firebase Auth. The implicit flow works fine but the code flow fails with either of the following errors :

Malformed response cannot be parsed from oidc.<provider_name> for CODE_EXCHANGE

or

Invalid OAuth response from oidc.<provider_name>

On investigation I found that the last call where this failure happens is this :

curl --location --request POST 'https://www.googleapis.com/identitytoolkit/v3/relyingparty/verifyAssertion?key=<web_key>' \
--header 'Content-Type: text/plain' \
--data-raw '{
    "returnSecureToken" : true,
    "requestUri" : "https://<firebase_project_name>.firebaseapp.com/__/auth/handler?state=<state_goes_here>&code=<authcode_flow_code>",
    "postBody" : "providerId=oidc.<provider_name>",
    "sessionId" : "<random_session_id>",
    "autoCreate" : true,
    "returnIdpCredential" : true
}'

Response for this request comes as :

{
    "error": {
        "code": 400,
        "message": "INVALID_IDP_RESPONSE : Invalid OAuth response from oidc.<provider_name>",
        "errors": [
            {
                "message": "INVALID_IDP_RESPONSE : Invalid OAuth response from oidc.<provider_name>",
                "domain": "global",
                "reason": "invalid"
            }
        ]
    }
}

I tried checking Firebase logs but no helpful information available. However, I did noticed that Firebase exchanged this code for the id_token successfully from Azure B2C before failing and returning 400 error.

Any help in this regard?