How to copy parquet file and overwrite the old one on HDFS

user_5983914 0
Viewed 1307

My question is suppose we have two parquet files on HDFS: file1.parquet, and file2.parquet in different paths, and I want to copy file1.parquet to replace file2.parquet.

I used the command hdfs dfs -cp -f /path1/file1.parquet /path2/file2.parqet, there was not error thrown, but the file2.parquet was not changed at all. I have to delete file2.parquet first and then copy file1.parquet.

Any idea? Thanks

hdfs parquet apache-spark
1 Answers

Check below commands.

hdfs dfs -cp -f /path1/file1.parquet /path2/file2.parqet // Overwitten file1 content in file2 content.
hdfs dfs -mv /path2/file2.parquet /path2/file1.parqet // For rename file from file2.parqet to file1.parqet
user_85934140
Related