AWS SAM Model fails while creating API with Custom domain name using cross account hosted zone

Problem Statement

My R53 Domain name and Hosted zone is in Account A. I want to create an API in Account B (Thru SAM Model) which will have a custom domain name declared in Account A.

Here is my sample SAM Template

AWSTemplateFormatVersion : '2010-09-09'
Globals:
  Api:
    OpenApiVersion: 3.0.1
Transform: AWS::Serverless-2016-10-31
Resources:
  ApiGatewayApi:
    Type: AWS::Serverless::Api
    Properties:
      Domain:
        CertificateArn: pCertificateArn
        BasePath:
          - pVersion
        DomainName: pCustomDomainName
        Route53:
          HostedZoneId: pHostedZoneId
      EndpointConfiguration: REGIONAL
      StageName: pStageName
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri:
      Handler: hello.lambdaHandler
      Runtime: nodejs12.x
      Events:
        AuthApi:
          Type: Api
          Properties:
            Path: /hello-main
            Method: GET
            RestApiId: !Ref ApiGatewayApi

Im executing this SAM template via a Pipeline, here is the code for the deploy role of the pipeline. This will be executed in Account B

 rDeployProjectRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub '${AWS::StackName}-DEPLOYPROJECT-ROLE-${pEnv}'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service: 
              - cloudformation.amazonaws.com
          Action: sts:AssumeRole
      Policies:
      - PolicyName: !Sub '${AWS::StackName}-DEPLOYPROJECT-POLICY-${pEnv}'
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
<highlighting only the necessary assumed role>
            - Effect: Allow
              Action:
                - sts:AssumeRole
              Resource: !Ref pRoute53AssumedRoleArn  

Now in my Account A, i have created a role to give access to the Account B like this

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  CreateRoute53Role:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal: 
              AWS:
                - arn:aws:iam::<Account B>:root
            Action:
              - sts:AssumeRole
      Path: "/"
      Policies:
        - PolicyName: Route53Access
          PolicyDocument:
            Version: 2012-10-17
            Statement:
            - Effect: Allow
              Action:
                - route53:*
              Resource: 
                - arn:aws:route53:::hostedzone/XXX

Everytime i run the above code, it fails with this error in the cfn stack -

API: route53:GetHostedZone User: arn:aws:sts::XXX:assumed-role/XXX-DEPLOYPROJECT-ROLE-dev/AWSCloudFormation is not authorized to access this resource

Interesting when i look up the role (XXX-DEPLOYPROJECT-ROLE-dev) in IAM and look at the access advisor, it says that my pRoute53AssumedRoleArn is not even accessed. Im not sure what im doing wrong.

Also, if i were to create my API through the SAM template in Account A, it gets created perfectly fine, with the custom domain name, just the way i want it.

It seems problematic only with the cross account domain name access.