Is this data representation here exact for One-Hot Encoding?

user_15632586 0
Viewed 108

I am trying to encode the mushroom dataset here (https://www.kaggle.com/uciml/mushroom-classification) using One-Hot Encoding. Here is the code that I used (in Python) for the encoding:

from sklearn.preprocessing import OneHotEncoder
second_df = OneHotEncoder(handle_unknown='ignore').fit_transform(new_df)
print(second_df)

The result for my code is in this image, and that makes me quite confusing: Result for the encoding.

Is this result the right representation for my One-Hot Encoding? If not, what shall I do to fix the code?

one-hot-encoding scikit-learn python
1 Answers

The output looks a bit unusual because OneHotEncoder returns a sparse matrix by default:

OneHotEncoder(*, categories='auto', drop=None, sparse=True, dtype=<class 'numpy.float64'>, handle_unknown='error')

Sparse output is interpreted as (row, col) non_zero_value, where all the unlisted coordinates are zero:

(0, 1)    1.0         # value 1.0 at row 0, col 1
(0, 7)    1.0         # value 1.0 at row 0, col 7
...

To get a dense array instead,

  • either set sparse=False:

    OneHotEncoder(sparse=False).fit_transform(new_df)

  • or chain toarray:

    OneHotEncoder().fit_transform(new_df).toarray()

Output:

array([[0., 0., 0., ..., 0., 0., 1.],
       [0., 0., 0., ..., 0., 1., 0.],
       [1., 0., 0., ..., 0., 1., 0.],
       ...,
       [0., 0., 1., ..., 0., 1., 0.],
       [0., 0., 0., ..., 0., 0., 1.],
       [0., 0., 0., ..., 0., 1., 0.]])
user_131383640
Related