I'm converting CSVs to parquets and I need the schema to remain consistent. I have datetimes and I'd like to only save as a date. df.column.dt.date seems to work as long as there is at least one datetime, but when the column is all NaT, it remains as a datetime64[ns] type. Code example:
df = pd.DataFrame({
"dt1":["2020-02-04","2021-05-02"],
"dt2":[pd.NaT,pd.NaT],
"dt3":[pd.NaT,"2020-03-26"]
})
df["dt1"] = pd.to_datetime(df["dt1"])
df["dt2"] = pd.to_datetime(df["dt2"])
df["dt3"] = pd.to_datetime(df["dt3"])
As you can see here, for the column with two datetimes and the column with a datetime and NaT, it is converted into an object (datetime.date), but when all values are NaT, it remains as a datetime64[ns] type. Is there a way to convert to a date type that will be consistent for columns with all NaT as well?
