Currently, when I use the paritionBy() to write to HDFS:
DF.write.partitionBy("id")
I will get output structure looking like (which is the default behaviour):
../id=1/
../id=2/
../id=3/
I would like a structure looking like:
../a/
../b/
../c/
such that:
if id = 1, then a
if id = 2, then b
.. etc
Is there a way to change the filename output? If not, what is the best way to do this?