how to append a dataframe to existing csv file and remove the duplicates data

Question

i have a dataframe data and csv data as following

i want to append the dataframe data to the csv file.however ,there are duplicates between the two data.so how to remove the duplicates and append new data to csv file.

user_2901002 · Answer

I think you need read_csv with DataFrame.append, then drop_duplicates and last to_csv:

pd.read_csv('file').append(df).drop_duplicates().to_csv('file')

how to append a dataframe to existing csv file and remove the duplicates data

1 Answers