Small part file issue is a problem for HDFS storage as it increases the Metadata load for Name node. To reduce this problem can we use coalesce to reduce the partitions and then save the dataframe in HDFS?
0 Answers
Related
Small part file issue is a problem for HDFS storage as it increases the Metadata load for Name node. To reduce this problem can we use coalesce to reduce the partitions and then save the dataframe in HDFS?