Multiple column selection based on values

user_10617357 0
Viewed 66

I have the following data frame: 

df = pd.DataFrame({ 'Group' : [1,1,1,2,2,2,2],
               'Type' : ["High","Medium","Low","High","Medium","Low","Low"],
               'set_0' :["a","a","a","a","a","a","a"],
               'set_1' :["b","b","b","c","c","c","d"],
               'set_2' :["e","e","e","NULL","NULL","f","f"],
               'set_3' :["g","g","NULL","NULL","NULL","NULL","NULL"],
               'set_4' :["NULL","NULL","NULL","NULL","NULL","NULL","NULL"],
               'set_5' :["NULL","NULL","NULL","NULL","NULL","NULL","NULL"],
               'set_6' :["h","h","NULL","NULL","NULL","NULL","NULL"]
                                 })

I want to drop some of "set_" columns. If a "set_" related column has all "NULL" values I don't want the code to keep them. I only want to keep set_ columns where it contains at least one non-"NULL" value.

How can I handle this without hardcoding it?

pandas dataframe indexing python-3.x python
1 Answers

First select object dtype series and test versus your specified string. Then use pd.DataFrame.loc with Boolean indexing or pd.DataFrame.drop:

idx = df.select_dtypes(['object']).eq('NULL').all()

df = df.loc[:, ~df.columns.isin(idx[idx].index)]

# alternative:
# df = df.drop(idx[idx].index, 1)

print(df)

   Group    Type set_0 set_1 set_2 set_3 set_6
0      1    High     a     b     e     g     h
1      1  Medium     a     b     e     g     h
2      1     Low     a     b     e  NULL  NULL
3      2    High     a     c  NULL  NULL  NULL
4      2  Medium     a     c  NULL  NULL  NULL
5      2     Low     a     c     f  NULL  NULL
6      2     Low     a     d     f  NULL  NULL
user_92095460
Related