In my work we have 2 cloudera clusters (dev, prod)
On the dev cluster if I write:
spark.read('/path/filename.csv')
It reads it from local filesystem (i.e. file://path/filename.csv)
On the prod cluster the same line reads from hdfs. (i.e. hdfs:///path/filename.csv)
My question is, is there a certain configuration that lets this happen because I cant understand why the same command behaves differently.