I need to create a script that accepts as input: a very large csv file, a starting time, a window size, a shift time, and a delta time.
Dask Index Structure:
npartitions=49
2022-06-30 19:43:30 datetime64[ns]
2022-07-01 01:46:43 ...
...
2022-07-13 04:17:22 ...
2022-07-13 10:50:46 ...
Name: Timestamp, dtype: datetime64[ns]
Dask Name: sort_index, 196 tasks
The code should create csv files that spans window size amount of time (for example 20 minutes) that starts at starting time and ends at (starting time + window size) time (for example starting at midnight, 0:00 to 0:20 for a 20 minute window time).
The file should have entries every delta time. For example, if delta time = 10 seconds, then there should be 10-second increments in the csv file.
Once a csv file spanning the window size time is created, a new csv file should be created that starts at starting time + shift time and ends windows size time later. For example, the first csv file may cover minutes 0-20 and the second csv file covers minutes 2-22.
csv files should continue until one of them reaches ending time.
Note: There is a unique measurement at every single second of the input time series.