When using support vector regression (SVR) from Scikit-learn, the advice is to scale the data, as Support Vector Machine algorithms are not scale invariant (see user guide). This can be done via standardScaler() or MinMaxScaler().
However, should I also scale the possible values for the parameter Epsilon?
Epsilon specifies the epsilon-tube within which no penalty is associated in the training loss function with points predicted within a distance epsilon from the actual value. If I understand correctly, this means that if I want to build a model to predict housing prices and I allow that my predictions are 5000 euro off (i.e. higher or lower), then my epsilon could be "5000".
As the SVR algorithm does not know the original housing prices, only the scaled ones, it makes sense to also scale the epsilon values using the same procedure. Is this correct?
For example, If I scale housing prices between 0 and 1.
prices = np.array([[0], [100000], [150000], [200000], [180000]])
scaler_x = MinMaxScaler(feature_range=(0, 1))
prices_scaled = scaler_x.fit_transform(prices)
print(prices_scaled )
>>> [[0. ]
[0.5 ]
[0.75]
[1. ]
[0.9 ]]
Should I use same scaling on epsilon (i.e. 5000 --> 0.025)?
cv = ShuffleSplit(n_splits = 10, test_size = 0.25, random_state = 0)
grd = GridSearchCV(estimator=SVR(kernel='linear'),
param_grid={'C': c_list,
'epsilon': [0.025]}, #scaled epsilon instead of 5000?
cv=cv, scoring='r2')
grid_result = grd.fit(predictors_scaled, prices_scaled )
Note that the predictors are scaled as well, but using another scaling.
In this example both predictors and response are scaled, but there is no mentioning that epsilon should be scaled as well.
If epsilon does not have to be scaled, which values should I then use?