When to use "accuracy" string or tf.keras.metrics.Accuracy() in a classifier neural network model

user_13822114 0
Viewed 299

I'm learning about neural network and I have a question about metrics. In the book I'm reading it says I could use as "metrics" at model.compile() a string or the full name of the function like bellow. I'm using Google Colab Research.

model.compile(loss=tf.keras.losses.binary_crossentropy, 
          optimizer=tf.keras.optimizers.Adam(), 
          metrics=["accuracy"])

model.compile(loss=tf.keras.losses.binary_crossentropy, 
          optimizer=tf.keras.optimizers.Adam(), 
          metrics=[tf.keras.metrics.Accuracy()])

But when I run model.evaluate() I get always very different numbers like:

# metrics=tf.keras.metrics.Accuracy()
print(model.evaluate(X_test, y_test))
25/25 [==============================] - 0s 1ms/step - loss: 0.0084 - accuracy: 0.0000e+00
[0.008412548340857029, 0.0]

# metrics="accuracy"
print(model.evaluate(X_test, y_test))
25/25 [==============================] - 0s 1ms/step - loss: 0.0078 - accuracy: 1.0000
[0.00784504134207964, 1.0]

I know when I restart the model I get different random weights but always I'm getting this 0.0000e+00 when using the complete function path. I would like ot know why and when to use strings and the complete function name.

Bellow the code:

from  sklearn.datasets          import make_circles
from  sklearn.model_selection   import train_test_split

import tensorflow         as tf

n_samples = 1000
X, y = make_circles(n_samples, noise=0.03, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.relu, input_shape=(2,)))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.relu))
model.add(tf.keras.layers.Dense(1, activation=tf.keras.activations.sigmoid))

model.compile(loss=tf.keras.losses.binary_crossentropy, 
              optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), 
              metrics="accuracy")

hist = model.fit(X_train, y_train, epochs=100, verbose=0)

print(model.evaluate(X_test, y_test))
keras tensorflow neural-network python
1 Answers

It seems that 'accuracy' actually corresponds to tf.keras.metrics.BinaryAccuracy() in this case, see the code below.

from sklearn.datasets import make_circles
from sklearn.model_selection import train_test_split
import tensorflow as tf

n_samples = 1000
X, y = make_circles(n_samples, noise=0.1, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.relu, input_shape=(2, )))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.relu))
model.add(tf.keras.layers.Dense(1, activation=tf.keras.activations.sigmoid))

model.compile(
    loss=tf.keras.losses.binary_crossentropy,
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
    metrics=['accuracy', tf.keras.metrics.BinaryAccuracy(name='binary_accuracy')]
)

model.fit(X_train, y_train, epochs=100, verbose=1)
# Epoch 1/100
# 25/25 [==============================] - 0s 678us/step - loss: 0.3403 - accuracy: 0.8562 - binary_accuracy: 0.8562
# Epoch 2/100
# 25/25 [==============================] - 0s 792us/step - loss: 0.3401 - accuracy: 0.8625 - binary_accuracy: 0.8625
# Epoch 3/100
# 25/25 [==============================] - 0s 748us/step - loss: 0.3406 - accuracy: 0.8575 - binary_accuracy: 0.8575
# ....

print(model.evaluate(X_test, y_test))
# [0.5954229235649109, 0.7200000286102295, 0.7200000286102295]
user_119890810
Related