Encode character columns as ordinal but keep numeric columns the same

user_8364914 0
Viewed 773

I have a data frame like the following:

test = {"viral": "pos", "Status": "positive", "Age": 59, "score": 5}
test2 = {"viral": "neg"}
df = pd.DataFrame.from_dict([test, test2])

I want to encode the character columns (viral, Status) as ordinal numbers, but leave the numeric values alone. Desired output is a numpy array with the same column names.

If I use OrdinalEncoder from sklearn, it does not deal with NaN values. Even if there were not NaN values, it would still ordinal encode the numeric columns also however. I want to fill NaN values with 0 in character columns but keep NaN in numeric columns.

What is the easiest way to do this?

Desired output (in numpy array):

    Age    Status  score viral
0  59.0         1    5.0     1
1   NaN         0    NaN     0

Thanks! Jack

EDIT: I would also like a mapping from encoded value to original value like {i: dict(enumerate(v)) for i, v in enumerate(enc.categories_)} when using enc=OrdinalEncoder() (see Vectorize 2D character array column-wise)

scikit-learn pandas numpy python
2 Answers

Update, when there is NaN it will cat the code as -1 , if you need NaN you can convert back by using replace

df=pd.DataFrame([test,test2])
df.dtypes
Out[152]:
Age       float64
Status     object
score     float64
viral      object
dtype: object
listc=df.columns[df.dtypes=='object']

for x in listc:
    df[x]=df[x].astype('category').cat.codes

df
Out[156]: 
    Age  Status  score  viral
0  59.0       0    5.0      1
1   NaN      -1    NaN      0
user_79645270

Using LabelEncoder 

enc = sklearn.preprocessing.LabelEncoder()

mask = df.dtypes.eq(np.object)
df.loc[:, mask] = df.loc[:, mask].astype(str).apply(enc.fit_transform)

For you to be able to inverse_transform, you can create use a defaultdict of LabelEncoders. The key is your choice. I'd suggest using the column name: intuitive and simple

from collections import defaultdict
enc = defaultdict(sklearn.preprocessing.LabelEncoder)

mask = df.dtypes.eq(np.object)
df.loc[:, mask] = df.loc[:, mask].astype(str).apply(lambda s: enc[s.name].fit_transform(s))

    Status  viral
0   1       1
1   0       0

To inverse_transform

df.loc[:, mask].apply(lambda s: enc[s.name].inverse_transform(s))

    Status      viral
0   positive    pos
1   NaN         neg
user_25356110
Related