Keras to_categorical outputs more categories than actual labels

user_7386830 0
Viewed 1368

I have a Dataframe which consists of 11 different categories of integer values. It's similar to different integer codes such as:

"valy" is the name of the dataframe, ErrorCode is the column with target (multiclass categories).

valy.ErrorCode.unique()
array([ 9, 14,  1, 17,  6,  5, 24, 23, 16, 15, 11], dtype=int64)

valy.ErrorCode.nunique()
11

When I attempt to apply Keras to_categorical function, I expect to obtain 11 classes of one-hot encoded values in each row. However, it is resulting in 25 different classes. I'm not sure why this happens.

to_categorical(valy)[:1]
array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0.]], dtype=float32)

I also tried the following, resulting in an error:

tf.keras.utils.to_categorical(valy, num_classes=11, dtype='int64')

     47   n = y.shape[0]
     48   categorical = np.zeros((n, num_classes), dtype=dtype)
---> 49   categorical[np.arange(n), y] = 1
     50   output_shape = input_shape + (num_classes,)
     51   categorical = np.reshape(categorical, output_shape)

IndexError: index 14 is out of bounds for axis 1 with size 11

But, this version seems to work:

tf.keras.utils.to_categorical(valy, num_classes=25, dtype='int64')

Even though, I don't really have 25 multiclass labels in the column. There are only 11 different classes.

Is there a way to fix this issue ?

one-hot-encoding keras tensorflow categories python
1 Answers

I solved my question earlier.

For e.g:

valy.ErrorCode.unique()

array([ 9, 14,  1, 17,  6,  5, 24, 23, 16, 15, 11], dtype=int64)

Although there are 11 different classes in the above array, the way Keras (to_categorical) encodes categories is by taking the max value from the provided array and creating as many classes as there are in terms of integers.

In this example, the max value is 24. Hence, the number of created classes as one-hot-encoding is by starting from 0 and all the way until 24 (in this case 25 values).

user_73868300
Related