var hFile = sc.textFile("hdfs://localhost:9000/ex1/cen.csv") Input path does not exist error

user_13568513 0
Viewed 264

I am trying to access hadoop file in spark but I am getting this error

org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://localhost:9000/ex1/cen.csv
  at org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:287)

I am able to display the file in hadoop

hadoop dfs -cat ex1/cen.csv
apache-spark hadoop
3 Answers

I was able to solve the problem I tried the command hdfs dfs -ls / and used the directory path of folders shown in this iisting and it worked fine.I guess the issue was with the path.

user_135685130

Keep the hive-site.xml into conf folder of spark will resolve the issue !!

user_134603500

When you try 

hadoop dfs -cat ex1/cen.csv

the path to read the file in HDFS is 

/user/.../ex1/cen.csv

But if you try 

hadoop dfs -cat /ex1/cen.csv

Directory /ex1 has to be placed in the root directory / What you are trying to do with

 hdfs://localhost:9000/ex1/cen.csv

is to read from the root directory, and I think, your file isn't there because

/ex1/cen.csv

ex1/cen.csv

are different paths.

user_85714980
Related