How to access Appsync API from lambda function with AWS_IAM authorization

I have a GraphQL mutation that is protected by AWS_IAM authorization. I am using AWS_COGNITO_POOL as the main authorizer and AWS_IAM as the additional authorizer. I am using the serverless-appsync-plugin to integrate with AWS Appsync. This is the mutation:

updateUser(user_id: String!): String
@aws_iam

I need to call this mutation from the lambda function (python). Till now I have created IAM ROLE in the serverless config:

    UpdateUserMutationRole:
      Type: AWS::IAM::Role
      Properties:
        RoleName: ${self:provider.stage}-UpdateUserMutationRole
        AssumeRolePolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Principal:
                Service:
                  - lambda.amazonaws.com
              Action:
                - sts:AssumeRole
        Policies:
          - PolicyName: ${self:provider.stage}-UpdateUserMutationLambdaPolicy
            PolicyDocument:
              Version: "2012-10-17"
              Statement:
                - Effect: Allow
                  Action:
                    - appsync:GraphQL
                  Resource:
                    - <GraphQL API>

I have tried to assign this to lambda function but it does not work:

  updateUserMutation:
    handler: <path-to-handler>
    role: UpdateUserMutationRole

Inside the lambda function I am doing this:

    session = requests.Session()
        session.auth = AWS4Auth(
            <Access_key>,
            <Access_Secret>,
            <region>,
            'appsync'
        )
        response = session.request(
            url=APPSYNC_API_ENDPOINT_URL,
            method='POST',
            json={'query': query, "variables": variables}
        )

However, I keep getting 'Session Token is invalid' or 'Unauthorized exception'. I am completely clueless as of now. Can anyone here point me here in correct direction. Thanks