I have a parquet file with a date field in it called BusinessDate. When I import it to a dataframe it automatically determines the field BusinessDate being a date (datetime64[ns, UTC]). However, since the format of this BusinessDate field is YYYY-MM-DD, some of these dates were incorrectly imported. E.g. 2013-02-01 should be the first of february 2013 while it actually was interpreted as the second of January 2013. Is it possible to set the correct format of the BusinessDate field when importing the parquet file?
Initially I used:
df.read_parquet('data.parquet')
If I had a csv file my solution would be:
custom_date_parser = lambda x: datetime.strptime(x, '%Y-%m-%d')
df.read_csv('data.csv',parse_dates=['BusinessDate'], date_parser=custom_date_parser)
However, when I try a comparable code to try to fix the date issue, it gives an error:
custom_date_parser = lambda x: datetime.strptime(x, '%Y-%m-%d')
df.read_parquet('data.parquet',parse_dates=['BusinessDate'], date_parser=custom_date_parser)
The error is caused due to the fact that the read_parquet function doesn't have a parse_dates or date_parser attribute which e.g. a read_csv function does have.
So my question is: How do I import a parquet file in pandas so that the 'BusinessDate' field is imported correctly as a date in the correct format which is YYYY-MM-DD in my case. Or in case this not possible with pandas read_parquet function, is it possible to import the 'BusinessDate' field as a string field in a pandas dataframe so that I can change it afterwards.