How to one-hot-encode from a pandas column containing a list?

user_5858873 0
Viewed 31787

I would like to break down a pandas column consisting of a list of elements into as many columns as there are unique elements i.e. one-hot-encode them (with value 1 representing a given element existing in a row and 0 in the case of absence).

For example, taking dataframe df

Col1   Col2         Col3
 C      33     [Apple, Orange, Banana]
 A      2.5    [Apple, Grape]
 B      42     [Banana]

I would like to convert this to:

df 

Col1   Col2   Apple   Orange   Banana   Grape
 C      33     1        1        1       0
 A      2.5    1        0        0       1
 B      42     0        0        1       0

How can I use pandas/sklearn to achieve this?

sklearn-pandas scikit-learn pandas numpy python
6 Answers

You can use the functions explode (new in version 0.25.0.) and crosstab:

s = df['Col3'].explode()
df[['Col1', 'Col2']].join(pd.crosstab(s.index, s))

or in Python 3.7+:

df[['Col1', 'Col2']].join(pd.crosstab((s:=df['Col3'].explode()).index, s))

another approach using the method isin:

from itertools import chain

lst = sorted(set(chain.from_iterable(df['Col3'])))
s = pd.Series(lst, index=lst)
df.join(df.pop('Col3').apply(lambda x: s.isin(x)).astype(int))

Output:

  Col1  Col2  Apple  Banana  Grape  Orange
0    C  33.0      1       1      0       1
1    A   2.5      1       0      1       0
2    B  42.0      0       1      0       0
user_89736200
Related