MinMax Scaler and TensorFlow

user_10924836 0
Viewed 470

I am working with Sklearn and Tensorflow. I made a successful prediction and now I have a problem with inversion of values from prediction. I want to make an inversion because I want to see metrics from this model. At the beginning I made this transformation of data with Sklearn.

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler(feature_range = (0,1))
dataset = scaler.fit_transform(df)

type(dataset),dataset.shape
(numpy.ndarray, (425, 3))

After successful execution of TensorFlow model next step is prediction. I made prediction with lines of codes

trainPredict = model.predict(trainX)
testPredict = model.predict(testX)

trainPredict.shape,testPredict.shape
((406, 7, 1), (58, 7, 1))

type(trainPredict),type(testPredict)
(numpy.ndarray, numpy.ndarray)

So the next step is inversion. I try to do inversion with this line of code

# invert predictions
trainPredict_ = scaler.inverse_transform(trainPredict)
trainY_ = scaler.inverse_transform([trainY])
testPredict_ = scaler.inverse_transform(testPredict)
testY_ = scaler.inverse_transform([testY])

After execution of last block of code I get this error:

ValueError: Found array with dim 3. Estimator expected <= 2.
scikit-learn tensorflow numpy python
1 Answers

You don't need to scale target variables. Look at this Kaggle discussion, all participants are against scaling target variable. I also don't scale target variable. But if we look at this stack overflow discussion, here some people are saying to scale target and most people are against it. Even if you search it on internet, most of the people will be against scaling target value.
If you scale your target values, you might get lower rmse, but that's only because the target are very close to each other. So that lower rmse is a kind of illusion. It does not show the real performance of our model. In other words, scaling target does not have any great impact on our model.

So, my suggestion for you is to first separate features and target and then scale only features, not target.

y = train['target_column_name']
X = train.loc[:, train.columns != 'target_column_name']

scaler = MinMaxScaler()
scaler.fit(X)
X = scaler.transform(X)
test = scaler.transform(test)

So, in short, I want to say that don't scale target. Let it be as it is.

Or if you want to scale target, scale it separately. So that, after prediction, inverse scaling is possible.

user_144255010
Related