Do i need to provide .pem file to CircleCI Environment every time i connect to AWS EC2 via SSH?

I deploy to aws ec2 using CircleCI (CI/CD pipeline)and i save pem file(private AWS EC2 instance key file) to CircleCI environment. I connect via SSH using this command, something like

ssh -i CIRCLECI_PEMFILE ubuntu@myinstance.compute.amazonaws.com

But i got 2 problems

1. It doesn't work!
2. Even if it worked but i still think that i am doing something wrong that it seem less secure. How can i connect to AWS EC2 but still keep my pem file protecced (i mean may be something like decrypt)?

I am too new in ci/cd and sorry for any confused. Thank you very much!