I am pandas to read a csv file then export it to Parquet partitioned by date, it works great
import pandas as pd
import datetime
df = pd.read_csv('box.csv',parse_dates=True)
df['SETTLEMENTDATE'] = pd.to_datetime(df['SETTLEMENTDATE'])
df['Date'] = df['SETTLEMENTDATE'].dt.date
df.to_parquet('nem.parquet',partition_cols=['Date'],allow_truncated_timestamps=True)
When I run the script again, it will generate a new files

is there an Option in Pandas to overwrite the file instead, and add a new file only when there is a new data ?
