Conversion of timestamp in Dask

user2woI 0
Viewed 34

Below is the code I did in Pandas, but I need to do it in Dask.

df['converted'] = (df['timestamp'] - pd.Timestamp("1960-02-02").tz_localize(tz=df['timestamp][0].tz)) / pd.Timedelta('5s')

Here is the data frame df:

timestamp
2016-01-11 12:33:28.616000+00:00
2016-01-11 12:34:46.447000+00:00
2016-01-12 09:32:11.954000+00:00
2016-01-12 09:32:11.983000+00:00
2016-01-12 09:32:11.993000+00:00

I have no idea how to convert this because some functions are not available in DASK. All I can find is: https://docs.dask.org/en/stable/generated/dask.dataframe.to_datetime.html#to-datetime-tz-examples

Could you help me how can I do this in DASK?

dask pandas python
1 Answers

If your task is exactly as you describe, threre shouldn't be any problem to use dask here. All we have to do is subtract from DataFrame some constant value and divide the result by other one:

import pandas as pd
import dask.dataframe as dd
from io import StringIO

data = '''timestamp
2016-01-11 12:33:28.616000+00:00
2016-01-11 12:34:46.447000+00:00
2016-01-12 09:32:11.954000+00:00
2016-01-12 09:32:11.983000+00:00
2016-01-12 09:32:11.993000+00:00'''

df = pd.read_csv(StringIO(data), parse_dates=[0])

x = pd.Timestamp("1960-02-02").tz_localize(tz=df['timestamp'][0].tz)
y = pd.Timedelta('5s')

ddf = dd.from_pandas(df, npartitions=1)
output = (ddf - x) / y

df['converted'] = output.compute()
print(df)
Vitalizzare0
Related