Use OrdinalEncoder in a pipeline

user_15633003 0
Viewed 2013

I want to use sklearn OrdinalEncoder in a pipeline while making sure the right ordering of categories is made. How can I do that?

I have for example the features "Education_Level" and "Income_Category" I want to encode as follows:

"Education_Level": {
        "Uneducated": 1,
        "High School": 2,
        "College": 3,
        "Graduate": 4,
        "Post-Graduate": 5,
        "Doctorate": 6
},
    "Income_Category": {
        "Less than $40K": 1,
        "$40K - $60K": 2,
        "$60K - $80K": 3,
        "$80K - $120K": 4,
        "$120K +": 5
    }

Can someone help me on how to do that? I remind that I want to use OrdinalEncoder in a pipeline to perform the task. I can do it 'manually' with replace(), but since I am creating a pipeline, I need the encoder object (at least I guess).

scikit-learn python
2 Answers

You can specify the OrdinalEncoder categories parameter during its initialization. This will ensure that your categories have the right ordinal order.

You can do as follow:

from sklearn.preprocessing import OrdinalEncoder
from sklearn.compose import make_column_transformer

import pandas as pd

df = pd.DataFrame({
    'Education_Level': ['Uneducated', 'High School', 'College', 'Post-Graduate', 'Doctorate', 'Graduate'],
    'Income_Category' : ['Less than $40K','Less than $40K', '$40K - $60K', '$60K - $80K', '$80K - $120K', '$120K +']
    }
)


cat_education_lvl = [['Uneducated', 'High School', 'College', 'Graduate', 'Post-Graduate', 'Doctorate']]
income_cat = [['Less than $40K', '$40K - $60K', '$60K - $80K', '$80K - $120K', '$120K +']]
transformer = make_column_transformer(
    (OrdinalEncoder(categories=cat_education_lvl), ['Education_Level']),
    (OrdinalEncoder(categories=income_cat), ['Income_Category']),
)

transformer.fit_transform(df)

This transformer will return the following output:

array([[0., 0.],
       [1., 0.],
       [2., 1.],
       [4., 2.],
       [5., 3.],
       [3., 4.]])

EDIT: If you want to have the output as a pandas DataFrame the easiest way to have it is to recreate it as follows:

df_transformed = pd.DataFrame(transformer.fit_transform(df), columns=df.columns)
user_45746330

In another way you can do like this fast make two list For one is numeric columns & another one is categorical Then start pipeline like this: Also in this pipeline you fill in missing value numeric columns with mean & cat columns in most_frequent.

num_trans = Pipeline(steps = [('imputer', SimpleImputer(strategy='mean'))])
cat_trans = Pipeline(steps = [('imputer', SimpleImputer(strategy = 'most_frequent')),
                                       (('odi', (OrdinalEncoder(handle_unknown="use_encoded_value", unknown_value = None))))])
transformer = ColumnTransformer(transformers=[("num", num_trans, numerics_col_list),
                                           ("cat", cat_trans, cat_col_list)])

If you want get-back in pandas then

transform_data = pd.DataFrame(transformer.fit_transform(df), columns = (numerics_col_list+cat_col_list))

Its perfectly work for... Thx...

user_184748430
Related