Sklearn Pipelines: Value Error - Expected number of features

user_4777141 0
Viewed 187

I created a pipeline that basically loops over models and scalers and performs recursive feature elimination (RFE) as follows:

def train_models(models, scalers, X_train, y_train, X_val, y_val):
  best_results = {'f1_score': 0}

  for model in models:
    for scaler in scalers:
        for n_features in list(range(
            len(X_train.columns), 
            int(len(X_train.columns)/2), 
            -10
        )):
            rfe = RFE(
                estimator=model, 
                n_features_to_select=n_features, 
                step=10
            )
            
            pipe = Pipeline([
                ('scaler', scaler), 
                ('selector', rfe),
                ('model', model)
            ])

            pipe.fit(X_train, y_train)
            
            y_pred = pipe.predict(X_val)
            results = evaluate(y_val, y_pred) #Returns a dictionary of values
            results['pipeline'] = pipe
            results['y_pred'] = y_pred
            
            if results['f1_score'] > best_results['f1_score']:
                best_results = results
                print("Best F1: {}".format(best_results['f1_score']))
        
  return best_results

The pipeline works fine inside the function and is able to predict and score the results properly.

However, when I call pipeline.predict() outside the function, e.g.

best_result = train_models(models, scalers, X_train, y_train, X_val, y_val)
pipeline = best_result['pipeline']
pipeline.predict(X_val)

I get the following error: enter image description here

Here is what pipeline looks like:

Pipeline(steps=[('scaler', StandardScaler()),
                ('selector',
                 RFE(estimator=LogisticRegression(C=1, max_iter=1000,
                                                  penalty='l1',
                                                  solver='liblinear'),
                     n_features_to_select=78, step=10)),
                ('model',
                 LogisticRegression(C=1, max_iter=1000, penalty='l1',
                                    solver='liblinear'))])

I'm guessing the model in the pipeline is expecting 48 features instead of 78, but I don't understand where the number 48 is coming from since n_features_to_select is set to 78 in the previous RFE step!

Any help would be greatly appreciated!

feature-selection scikit-learn machine-learning pipeline python
1 Answers

I do not have your data. But doing some math and guessing based on the info you have shared, 48 seems to be the last n_features that your nested loop tries. This makes me suspect that the culprit is a shallow copy. I suggest you change the following:

    pipe = Pipeline([
        ('scaler', scaler), 
        ('selector', rfe),
        ('model', model)
    ])

to

    pipe = Pipeline([
        ('scaler', scaler), 
        ('selector', rfe),
        ('model', copy.deepcopy(model))
    ])

and try again (after first doing an import copy too, of course).

user_54251720
Related