I have two time series file which is meant to be in CET / CEST. The bad one of them, does not write the values in a proper way. For the good csv, see here:
#test_good.csv
local_time,value
...
2017-03-26 00:00,2016
2017-03-26 01:00,2017
2017-03-26 03:00,2018
2017-03-26 04:00,2019
...
2017-10-29 01:00,7224
2017-10-29 02:00,7225
2017-10-29 02:00,7226
2017-10-29 03:00,7227
...
...everything works fine by using:
df['utc_time'] = pd.to_datetime(df[local_time_column])
.dt.tz_localize('CET', ambiguous="infer")
.dt.tz_convert('UTC').dt.strftime('%Y-%m-%d %H:%M:%S')
When converting the test_bad.csv to UTC I get an AmbiguousTimeError as the 2 hours in October are missing.
# test_bad.csv
local_time,value
...
2017-03-26 00:00,2016
2017-03-26 01:00,2017 # everything is as it should be
2017-03-26 03:00,2018
2017-03-26 04:00,2019
...
2017-10-29 01:00,7223
2017-10-29 02:00,7224 # the value of 2 am should actually be repeated PLUS 3 am is missing
2017-10-29 04:00,7226
2017-10-29 05:00,7227
...
Does anyone know an elegant way of how to still convert the time series file to UTC and add NaN columns for the missing dates in the new index? Thanks for your help.