Keras: How to obtain confidence of prediction class?

user_10434122 0
Viewed 6588

I'm working in Keras/TensorFlow. This is my Keras model:

model = Sequential()
model.add(Dense(512, input_shape=(max_words,)))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

After training step and test step, I'm coding a method that take the input (which i don't know his class) e this method returns the class prediction with level of confidence. Now this method returns only the prediction of class. This is the method:

def predict(input):
    try:
        x_prediction = tokenize.texts_to_matrix(input)
        q = model.predict(np.array([x_prediction[0],]))
        predicted_label = text_labels[np.argmax(q)]

        print("Prediction: " + predicted_label + "\n")

    except:
        return "Error"

What should I add in the method to get the confidence level of the respective prediction? I don't want use the confidence of variable 'q' but I want to use the Bayes Approach. How can I do? Thanks

keras tensorflow python
2 Answers

In Keras, model.predict() actually returns you the confidence(s). So in the code snippet, you may want to print q to see the entire array with all confidence levels.

np.argmax(x) gives you the argument(position) in the array where X has the max value.

user_61170170

The values in the vector q are probabilities for each class, which act as a confidence value, so you can just fetch the maximum value and return it as confidence.

But notice that these probabilities are produced by the model, and they might be overconfident unless you use a model that produces calibrated probabilities (like a Bayesian Neural Network).

user_3491300
Related