Cannot Locate the file in AWS EMR Notebook

Viewed 300

I have been trying to use some .txt, .csv files in an EMR Notebook but I cannot locate them.

I am trying to read via:

with open('file.txt', 'r') as f:
    notes = f.read()

Things I tried:

  • Uploaded the file by using JupyterHub UI. I can see the file but I cant read it from the path. I also checked the file using JupyterHub terminal.

  • Tried to read from s3 (lots of people got it working in this way):

    with open('s3://<repo>/file.txt', 'r') as f:

  • Copied the file to hdfs and hadoop in master node (in cluster) using both: hdfs dfs and hadoop fs . File is present in both directories.

However, I have no clue how I can reach the file in EMR Notebook.

Any ideas?

0 Answers
Related