Can LSTMs handle incredibly dense time-series data?

user_13981320 0
Viewed 83

I have 50 time series, each having at least 500 data points (some series have as much as 2000+ data points). All the time series go from a value of 1.089 to 0.886, so you can see that the resolution for each dataset comes close to 10e-4, i.e. the data is something like:
1.079299, 1.078809, 1.078479, 1.078389, 1.078362,... and so on in a decreasing fashion from 1.089 to 0.886 for all 50 time-series.

My questions hence, are:

  1. Can LSTMs handle such dense data?
  2. In order to avoid overfitting, what can be the suggested number of epochs, timesteps per batch, batches, hidden layers and neurons per layer?

I have been struggling with this for more than a week, and no other source that I could find talks about this specific case, so it could also help others.

lstm keras deep-learning time-series python
1 Answers

A good question and I can understand why you did not find a lot of explanations because there are many tutorials which cover some basic concepts and aspects, not necessarily custom problems.

  1. You have 50 time series. However, the frequency of your data is not the same for each time series. You have to interpolate in order to reach the same number of samples for each time series if you want to properly construct the dataset.
  2. LSTMs can handle such dense data. It can be both a classification and a regression problem, neural networks can adapt to such situations.
  3. In order to avoid overfitting(LSTMs are very prone to it), the first major aspect to take into consideration is the hidden layers and the number of units per layer. Normally people tend to use 256-512 by default since in Natural Language Processing where you process huge datasets they are suitable. In my experience, for simpler regression/classification problems you do not need such a big number, it will only lead to overfitting in smaller problems.

Therefore, taking into consideration (1) and (2), start with an LSTM/GRU with 32 units and then the output layer. If you see that you do not have good results, add another layer (64 first 32 second) and then the output layer.

Admittedly, timesteps per batch is of crucial importance. This cannot be determined here, you have to manually iterate through values of it and see what yields you the best results. I assume you create your dataset via sliding window manner; consider this(window size) also a hyperparameter to alter before arriving at the batch and epochs ones.

user_61170170
Related