Preparing dataset TimeSeries data

user_10582840 0
Viewed 55

So I am working on a project where I have some time series data that I want to predict. The problem is that my dataset consists of different water samples taken from a water source and there are in a single csv file.

My dataset looks kinda like this:

Date         Sample_Name        pH    temp    etc...

2009-01-01    ABC1              7.2    12
2009-01-02    ABC2              5.5    11
.
.
.
2015-01-05    ABC1              8.9    13
2015-01-05    ABC4              8.8    13

So ABC1 and ABC2 are different samples and have information recorded every month. What I want to do is feed ABC1 explicitly into the model, but I don't know how to do that. I can group samples by their names with this line of code:

abc1 = df.loc[df['Sample_Name'] == "ABC1"]

How can I feed this kind of data into a model? I did not decide on the final model but it will probably be an Encoder/decoder(with attention) or an LSTM.

Each Sample contains about 70 rows and I have over 100 samples.

pandas time-series
1 Answers

Let me give it a go as it is not entirely clear what the desired output is, but hopefully will steer you in the right direction

Load your example:

from io import StringIO
data = StringIO(
'''
Date         Sample_Name        pH    temp
2009-01-01    ABC1              7.2    12
2009-01-02    ABC2              5.5    11
2015-01-05    ABC1              8.9    13
2015-01-05    ABC4              8.8    13
'''
)
df = pd.read_csv(data, sep = '\s+')

Then we can use groupby method to create a dictionary, keyed on each sample name, with the corresponding value being the dataframe for that sample that could be fed into a model:

input_dict = {key:df.drop(columns = 'Sample_Name').reset_index(drop = True) for key, df in df.sort_values('Date').groupby('Sample_Name')}

You can access individual sample dfs by the name of the sample, for example with

input_dict['ABC1']

you get the corresponding df:

    Date        pH  temp
0   2009-01-01  7.2 12
1   2015-01-05  8.9 13
user_145514260
Related