I have an EMR cluster set up and running. I need to add a python package to my cluster, and instead of shutting it down and re-boostrapping with the new package, I was wondering if it were possible to ssh into each of the task nodes and pip install the package there. I can ssh into the master node, but I can't find the ip addresses for the associated task nodes.
1 Answers
Yes, you can SSH into the Core/Task nodes.
You will need to modify the Security Group to allow inbound SSH (port 22) connections.
I tried it myself by looking in the EC2 console to get the IP address of the Core/Task node. I then used SSH to connect, just like the master node.
Related