How to standardize (and invert predictions) in multivariate multistep LSTM implementation in Keras

user_10381195 0
Viewed 593

I've got a working (albeit unoptimised) implementation of a multi-variate, multi-step time series forecasting model using LSTMs using ticker data for NYSE: ECL. The implementation is here: https://github.com/mintgreenstrat/SO_LSTM/tree/master

I need to apply scaling because I'm planning to add in a bunch more features (like Volume) that are of a vastly different scale. Applying the min/max scaler on the same 4 feature example is simple enough, then training the model and making the prediction.

What I can't for the life of me do is get the prediction (yhat), X_test and y_test back into the right shape to inverse the scaling. I'd been trying to adapt an example but I just can't get it right and I'm getting lost in it all.

At this point, I'm really struggling to understand whether:
a) I've made an error in setting up the data in the right shape with time_series_to_supervised steps
b) It's just a simple change to the np.concatenate(yhat, X_test[:, -(n_features -1):], axis=1)
c) Both a & c
d) Something else

For what it's worth, my intuition says b)...

lstm keras python
1 Answers

I finally got a solution to this with the help of this post on r/MLQuestions sub reddit.

I'm now splitting the dataset into X, y and fitting a separate scaler to each before I reshape to 3D for input to LSTM. That means on the back end, I just have to reshape from 3D back to 2D in order to call the inverse_transform function. It seems I'd made this harder than it needed to be.

I updated the github repo in case that'd help anyone in the future.

user_103811950
Related