Random Forrest feature_importance, how to trace back source col?

user_12242625 0
Viewed 35

Situation

I'm having a sklearn.ensemble.RandomForestRegressor model and want to find the most important feature.

MWE

from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer

import pandas as pd

df = pd.DataFrame({
    "ColA":[1,2,3,4,5,6,7,8,9,10],
    "ColB":["CharA","CharB","CharC","CharD","CharE","CharF","CharG","CharH","CharI","CharJ"],
    "ColC":[132,1000,5,20,165,852,403,680,481,6],
    "ColD":[1,2,3,4,5,6,7,1,2,3],
    "ColE":[2,26,5,7,1,2,3,12,65,12]
})

num_attribs=["ColA", "ColC"]
cat_attribs=["ColB", "ColD", "ColE"]

full_pipeline = ColumnTransformer([
    ("num", MinMaxScaler(), num_attribs), 
    ("cat", OneHotEncoder(), cat_attribs) 
])

prep_df = full_pipeline.fit_transform(df)

prep_df.toarray()[0]

Output:

array([0.        , 0.12763819, 1.        , 0.        , 0.        ,
       0.        , 0.        , 0.        , 0.        , 0.        ,
       0.        , 0.        , 1.        , 0.        , 0.        ,
       0.        , 0.        , 0.        , 0.        , 0.        ,
       1.        , 0.        , 0.        , 0.        , 0.        ,
       0.        , 0.        ])

Please note: this is just demo code, I know if I'd leave out toarray it would be easy, but in my real data (~1Mio rows, ~70cols) it results in this format.

Problem

I want to trace back to the original column, but I have the situation that I used OneHotEncoder inside a ColumnTransformer. So there is no inverse_transform function and I'm having more columns than before encoding.

Question

If model.feature_importances_.argmax() says --> 15, how can I find out which source column that was?

scikit-learn random-forest python
1 Answers

You could use get_feature_names_out() on the transformers of your column transformer and index into that array. For your example, that would read like

idx = model.feature_importances_.argmax()
cols = [col for t in full_pipeline.transformers_ for col in t[1].get_feature_names_out()]
name = col[idx]

Note that for the one hot columns you would get names like columnname_categoryname, which might be even more interesting than just the column name itself.

user_140787580
Related