Converting categorical variables to numbers based on frequency in a single line

user_3285138 0
Viewed 1061

This is similar to LabelEncoder from scikit-learn, but with the requirement that the number value assignments occur in order of frequency of the category, i.e., the higher occurring category being assigned the highest/lowest (depending on use-case) number.

E.g. If the variable can take values [a, b, c] with frequencies such as

  Category 
0        a 
0        a 
0        a 
0        a 
0        a 
1        b 
1        b 
1        b 
1        b 
1        b 
1        b 
1        b 
1        b 
1        b 
1        b 
2        c 
2        c

a occurs 5 times, b occurs 10 times and c occurs 2 times. Then I want the replacements be done as b=1, a=2 and c=3.

scikit-learn pandas numpy python
3 Answers

See argsort:

df['Order'] = df['Frequency'].argsort() + 1
df

returns

  Category  Frequency  Order
0        a          5      3
1        b         10      1
2        c          2      2
user_19538000

If you are using pandas, you can use its map() method:

import pandas as pd
data = pd.DataFrame([['a'], ['b'], ['c']], columns=['category'])

print(data)

  category
0        a
1        b
2        c

mapping_dict = {'b':1, 'a':2, 'c':3}

print(data['category'].map(mapping_dict))

0    2
1    1
2    3

LabelEncoder uses np.unique to find the unique values present in a column which returns values in alphabetically sorted order, so you cannot use the custom ordering in it.

user_33749960

As suggested by @Vivek Kumar, I used the map functionality, using a dict of the sorted column values as key and their position as value:

data.Category = data.Category.map(dict(zip(data.Category.value_counts().index, range(1, len(data.Category.value_counts().index)+1))))

Looks a bit dirty, would be much better to divide it into a couple of lines like this:

sorted_indices = data.Category.value_counts().index
data.Category = data.Category.map(dict(zip(sorted_indices, range(1, len(sorted_indices)+1))))

This is the closest I have to my requirement. The output looks like this:

    Category
0          2
1          2
2          2
3          2
4          2
5          1
6          1
7          1
8          1
9          1
10         1
11         1
12         1
13         1
14         1
15         3
16         3
user_32851380
Related