Mnist: get confusion matrix

user_2651073 0
Viewed 8803

I try to get the confusion matrix of mnist dataset.

It's my code:

mnist = tf.keras.datasets.mnist

(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(512, activation=tf.nn.tanh),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])


model.fit(x_train, y_train, epochs=1, callbacks=[history])

test_predictions = model.predict(x_test)


# Compute confusion matrix
confusion = tf.confusion_matrix(y_test, test_predictions)

The problem is that test_prediction is a 10000 x 10 matrix, while y_test is 10000 x 1 matrix. In fact, the neural net doesn't provide an output for every test sample. How can I calculate confusion matrix for such cases?

And how can I then present the confusion matrix? Can I use the sci-kit library for this purpose?

confusion-matrix keras tensorflow python
2 Answers

This is probably because your prediction consist of probabilities of all the possible classes. You need to pick the class with highest probability, which will result in the same dimension as y_test. You can use the argmax() method from numpy. It works something like this:

import numpy as np
a = np.array([[0.9,0.1,0],[0.2,0.3,0.5],[0.4,0.6,0]])
np.argmax(a, axis=0)
array([0, 2, 1])

You can use sklearn for generating confusion matrix. Your code will become something like this

from sklearn.metrics import confusion_matrix
import numpy as np

confusion = confusion_matrix(y_test, np.argmax(test_predictions,axis=1))
user_63845390

If you use .predict_classes method instead of just predict, you get a vector of classes with the highest probability.

Then, you can use confusion_matrix from sklearn.

test_predictions = model.predict_classes(x_test)

from sklearn.metrics import confusion_matrix

cm = confusion_matrix(y_true = y_test, y_pred = test_predictions)
print(cm)

The shape of test_predictions here is (10000,).

The printed result will be something like:

array([[ 967,    1,    1,    2,    0,    1,    5,    0,    2,    1],
   [   0, 1126,    3,    1,    0,    1,    1,    0,    3,    0],
   [   3,    2, 1001,    8,    1,    0,    3,    6,    8,    0],
   [   0,    0,    1, 1002,    0,    1,    0,    1,    5,    0],
   [   3,    1,    2,    2,  955,    2,    6,    1,    3,    7],
   [   3,    1,    0,   37,    1,  833,    9,    0,    6,    2],
   [   4,    3,    1,    1,    1,    3,  941,    0,    4,    0],
   [   2,    9,    8,    5,    0,    0,    0,  988,    8,    8],
   [   3,    1,    3,   10,    3,    2,    2,    3,  946,    1],
   [   3,    8,    0,   10,    8,    8,    1,    4,    5,  962]],
  dtype=int64)
user_103499600
Related