I am using UC Irvine ML Glass Identification dataset mentioned in a book "Applied Predictive Modelling".
I tried rudimentary logistic regression models using sklearn with and without the StandardScaler class and had 2.3% improvement in accuracy when cv =5.
I did use pipeline to not leak the data but i am not sure if i leaked it somehow.
lr = LogisticRegression(max_iter=10000, random_state=42)
results = cross_val_score(estimator=lr,cv=5,scoring="accuracy", X = glass_data.iloc[:,0:-1], y=glass_data.iloc[:,-1] )
# had to increase max iterations as data is unprocessed, gradient descent is too slow because of it.
np.mean(results)
above code had lower accuracy of .579
from sklearn.preprocessing import StandardScaler
# since we did not divide our data set and are using cross validation, we need to use a pipeline
# otherwise there will be data leakage
from sklearn.pipeline import Pipeline
clf = Pipeline([("scaler", StandardScaler()), ("lr", LogisticRegression( random_state=42))])
result = cross_val_score(clf, cv=5, scoring="accuracy", X=glass_data.iloc[:,0:-1], y=glass_data.iloc[:,-1])
np.mean(result)
2nd code block had accuracy of .6078
Also, i did try to increase max_iter to a very large number for case 1 but the cross_val_score average was always the same.
I am just confused why there is increase in accuracy with feature scaling? Shouldn't it just help with faster convergence?