Applying Standard Scaler to one model in Voting Classifier

user_10077354 0
Viewed 384

I have created an ensemble of various models like svc, LogisticRegression, LinearDiscriminantAnalysis and so on.

But the mlp classifier works better when I scale the data, but other models like LogisticRegression achieve less accuracy when I scale my data. So I want to scale data for only one model.

from sklearn import preprocessing
scaler = preprocessing.StandardScaler()
scaler.fit(X_train)
X_train_ = scaler.transform(X_train)
X_val_ = scaler.transform(X_val)

mlp = MLPClassifier(solver='lbfgs', alpha=1e-5,
                 hidden_layer_sizes=(5,2), random_state=1)
mlp.fit(X_train_, y_train)
y_pred = mlp.predict(X_val_)

Now when I create my voting classifier, I do not know how to use the scaled data for one model alone.

votingC = VotingClassifier(estimators=[('logistic_regression', lr),('SVC',svc),
                                       ('Catboost', cat),('ExtraTrees', et), ('LinearDiscriminantAnalysis', lda), 
                                       ('perceptron', p),('randomforest', r), ('nusvc', nusvc), ('knn', knn), 
                                       ('SGDClassifier', pac), ('bag', bag),('bnb', nc)], 
                           voting='hard', n_jobs=6, 
                           weights = [1.5,1.5,1,1,1,1,1,1,1,1,1,1])

votingC = votingC.fit(X_train, y_train)

Thanks a lot in advance !

scikit-learn classification machine-learning python
1 Answers

For the models that require scaling, you can build a pipeline, which then goes into the voting classifier. Example with scaled and unscaled support vector classifier:

from sklearn.ensemble import VotingClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

from sklearn.datasets import make_classification
from sklearn.pipeline import make_pipeline

X,y = make_classification(random_state=123)

scaled_svc = make_pipeline(StandardScaler(), SVC())

voting = VotingClassifier(estimators=[
    ('scaled_svc', scaled_svc),
    ('unscaled_svc', SVC())
])

v = voting.fit(X,y)
v.predict(X)

array([0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1,
       0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0,
       0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0,
       1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1,
       1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0])
user_68172450
Related