How to label categorical variables in Pandas in order?

user_6541950 0
Viewed 2189

I have a pandas dataframe with the following categorical variables as columns on the left, and their specific realizations on the right,

enter image description here

(apologies for low-res).

For a statistical regression, I want to label all of these categorical variables, so, for example, in LotShape, Reg becomes 0, IR1 becomes 1, IR2 2, and IR3 3. I found that scikit-learn's LabelEncoder can do the job, but there's a problem. Some of these categorical variables are implicitly ordinal, and 0, 1, ... need to be assigned to the right labels, and LotShape only happens to be in order there.

So my question is, how would I efficiently, in some order that I specify, label a large number of categorical variables?

scikit-learn pandas data-cleaning machine-learning python
2 Answers

One can use base python function list.index() for this.

Taking example lists from Coolio2654's answer: 

x= ['GA','TA','SA','TA','GA','TA','SA']
orderlist = ['GA','TA','SA']

Using map function: 

list(map(lambda a: orderlist.index(a), x))

Or list comprehension: 

[orderlist.index(a) for a in x]

Output for each of above is: 

[0, 1, 2, 1, 0, 1, 2]

Above can be put in a function which can be called for each categorical column.

user_35221300

May be this can help.

x= ['GA','TA','SA','TA','GA','TA','SA']

import numpy as np
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing.label import _encode
from sklearn.utils import column_or_1d
x = column_or_1d(x, warn=True)
classes_,encoded_values = _encode(x,uniques=np.array(['GA','TA','SA']),encode=True)
encoded_values, classes_

#(array([0, 1, 2, 1, 0, 1, 2]), ['GA', 'TA', 'SA'])

#comparing with labelencoder, which will sort the labels before encoding
le = LabelEncoder()

le.fit_transform(x),le.classes_

#
(array([0, 2, 1, 2, 0, 2, 1], dtype=int64),
 array(['GA', 'SA', 'TA'], dtype='<U2'))
user_63476290
Related