Sklearn ColumnTransformer + Pipeline = TypeError

user_15409341 0
Viewed 547

I am trying to use properly pipelines and column transformers from sklearn but always end up with an error. I reproduced it in the following example.

# Data to reproduce the error
X = pd.DataFrame([[1,  2 , 3,  1 ],
                  [1, '?', 2,  0 ],
                  [4,  5 , 6, '?']],
                 columns=['A', 'B', 'C', 'D'])

#SimpleImputer to change the values '?' with the mode
impute = SimpleImputer(missing_values='?', strategy='most_frequent')

#Simple one hot encoder
ohe = OneHotEncoder(handle_unknown='ignore', sparse=False)

col_transfo = ColumnTransformer(transformers=[
    ('missing_vals', impute, ['B', 'D']),
    ('one_hot', ohe, ['A', 'B'])],
    remainder='passthrough'
)

Then calling the transformer as follows:

col_transfo.fit_transform(X)

Returns the following error:

TypeError: Encoders require their input to be uniformly strings or numbers. Got ['int', 'str']
one-hot-encoding scikit-learn pandas preprocessor
3 Answers

ColumnTransformer applies its transformers in parallel, not in sequence. So the OneHotEncoder sees the un-imputed column B and balks at the mixed types.

In your case, it's probably fine to just impute on all the columns, and then encode A, B:

encoder = ColumnTransformer(transformers=[
    ('one_hot', ohe, ['A', 'B'])],
    remainder='passthrough'
)
preproc = Pipeline(steps=[
    ('impute', impute),
    ('encode', encoder),
    # optionally, just throw the model here...
])

If it's important that future missing values in A,C cause errors, then similarly wrap impute into its own ColumnTransformer.

See also Apply multiple preprocessing steps to a column in sklearn pipeline

user_104958930

It's giving you an error because OneHotEncoder accepts just one format of data. In your case, it's a mixture of numbers and object. To overcome this issue you can separate the pipeline after imputer and OneHotEncoder to use astype method on the output of the imputing . Something like:

ohe.fit_transform(imputer.fit_transform(X[['A','B']]).astype(float))
user_131313200

The error is not coming from the ColumnTransformer but from the OneHotEncoder object

col_transfo = ColumnTransformer(transformers=[
    ('missing_vals', impute, ['B', 'D'])],
    remainder='passthrough'
)

col_transfo.fit_transform(X)

array([[2, 1, 1, 3], [2, 0, 1, 2], [5, 0, 4, 6]], dtype=object)

ohe.fit_transform(X)

TypeError: argument must be a string or number

OneHotEncoder is throwing this error because the object get mixed type of values (int + string) to encode on the same column, you need to cast the float columns to string in order to apply it

user_112802890
Related