Why the accuracy and binary_accuracy in keras have same result?

user_806160 0
Viewed 5166

I created a simple model for binary classification with Keras. The code is:

    # create model
    model = Sequential()
    model.add(Dense(250, input_dim=1, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    # Compile model
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy', 'binary_accuracy'])

My purpose was check the result of accuracy and binary_accuracy is understand difference between them.

As Keras says binary_accuracy accuracy have threshold that default is .5, that `accuracy' haven't. When I test them with sample data the result is difference but in the train of model thy have same results in each epoch.

for this true and predicted sample I tested accuracy and binary_accuracy:

   y_true = [[1], [1], [0], [0]]
   y_pred = [[0.51], [1], [0], [0.4]]

For binary_accuracy is:

m = tf.keras.metrics.BinaryAccuracy()
m.update_state(y_true, y_pred)
m.result().numpy()

that result is: 1

For accuracy is:

m = tf.keras.metrics.Accuracy()
m.update_state(y_true, y_pred)
m.result().numpy()

and the result is: '.5'

But in the above model it is same for each of them in each epoch.

Edit

By changing the compile to this the result changed:

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy', tf.keras.metrics.BinaryAccuracy(threshold=.7)])

Why accuracy work like binary_accuracy with threshold=0.5 in model but not in out of model?

keras tensorflow classification
1 Answers

According to tf.keras.Model.compile() documentation:

When you pass the strings 'accuracy' or 'acc', we convert this to one of tf.keras.metrics.BinaryAccuracy, tf.keras.metrics.CategoricalAccuracy, tf.keras.metrics.SparseCategoricalAccuracy based on the loss function used and the model output shape. We do a similar conversion for the strings 'crossentropy' and 'ce' as well.

In your case it was transformed to BinaryAccuracy and hence result is the same.

However tf.keras.metrics.Accuracy is something completely different. If you read the documentation:

Calculates how often predictions equal labels.

which means it looks at unique values of y_pred and y_true and treats every unique value as a distinct label. In your case 0.51 and 0.4 are treated as a separate labels and because they are not equal to 1 and 0, respectively, you get 0.5

Apologies for marking this question as a duplicate at first, the behaviour is different in tf.keras than in keras package

user_97308620
Related