Bitbucket pipelines - SCP Deployment- Failing "Identity file /opt/atlassian/pipelines/agent/ssh/id_rsa_tmp not accessible: No such file or directory."

I am trying to add bitbucket pipelines to deploy my angular application to one of our servers.

I configured SSH on my server and I could fetch the Host's finger print under Known hosts in Bitbucket

Below is my YAML file.

image: node:14

pipelines:
  branches:
    master:
      - step:
          name: Building Test angular application
          caches:
            - node
          script:
            - echo "npm install in progress.."
            - npm install
            - echo "Installing angular/cli..."
            - npm install -g @angular/cli
            - echo "Starting the Build process.."
            - ng build
          artifacts:
            - dist/** # Save build for next steps
      - step:
          name: "Deployment"
          script:
            - pipe: atlassian/scp-deploy:0.3.3
              variables:
                USER: $USER
                SERVER: $SERVER
                REMOTE_PATH: '/c/testscp/'
                LOCAL_PATH: 'dist/*'
                SSH_KEY: $MY_SSH_KEY

The first step is running fine without any issues and I can see dist folder being added to the artifact however the second step is failing with the below error.

scp -rp -i /opt/atlassian/pipelines/agent/ssh/id_rsa_tmp dist/TestPipelineApplication <<USER>>@<<SERVERIP>>:/c/testscp/
Warning: Identity file /opt/atlassian/pipelines/agent/ssh/id_rsa_tmp not accessible: No such file or directory.
Load key "/root/.ssh/pipelines_id": invalid format
Permission denied, please try again.
Permission denied, please try again.
<<USER>>@<<SERVERIP>>: Permission denied (publickey,password,keyboard-interactive).

I never configured pipelines before so I am not completely sure what I am missing here.

Also, I looked into the below documentation but of no luck

https://bitbucket.org/atlassian/scp-deploy/src/1.0.1/README.md

Any help or suggestions are greatly appreciated.