how do I install python library on AWS EMR notebook?

user_6668398 0
Viewed 7902

I want to install additional libraries on AWS notebook (connected to EMR cluster), however I do not see any option to connect from Notebook to internet. If I do "pip install ", it always come back saying that network is not reachable. I am not sure which network need to be changed for network connection and library installation.

I did login to Jupyter terminal, and ping to google.com, which just timed out. I do not see any network / security group etc... configuration under Notebook section for making any relevant changes.

May be I need to take some additional steps?

jupyter amazon-emr jupyter-lab jupyter-notebook amazon-web-services
1 Answers

If you use PySpark kernel then you can install libraries using 

sc.install_pypi_package("celery")

Or by running 

sudo pip-3.6 install boto3

The following document has more details

If you use python 3 kernel, then only the packages are installed and there is no direct way to install extra libraries except uploading the python package to the notebook then using jupyterlab terminal to run

pip install package.tar.gz

user_43910110
Related