How can I pass logits to sigmoid_cross_entropy_with_logits before I fit and predict model?

user_2803344 0
Viewed 582

Since I need to train a model with multiple labels, I need to use loss function tf.nn.sigmoid_cross_entropy_with_logits. This function has two parameters: logits and loss.

Is parameter logitsis the value of predicted y? How can I pass this value before I compile model? I cannot predict y before I compile and fit model, right?

This is my code:

import tensorflow as tf
from tensorflow import keras

model = keras.Sequential([keras.layers.Dense(50, activation='tanh', input_shape=[100]), 
                            keras.layers.Dense(30, activation='relu'),
                            keras.layers.Dense(50, activation='tanh'),
                            keras.layers.Dense(100, activation='relu'),
                            keras.layers.Dense(8)])

model.compile(optimizer='rmsprop', 
              loss=tf.nn.sigmoid_cross_entropy_with_logits(logits=y_pred), labels=y),   # <---How to figure out y_pred here?
              metrics=['accuracy'])
model.fit(x, y, epochs=10, batch_size=32)
y_pred = model.predict(x)  # <--- Now I got y_pred after compile, fit and predict

I'm using tensorflow v2.1.0

multilabel-classification keras tensorflow machine-learning
1 Answers

These arguments (labels and logits) are passed to the loss function within Keras' implementation. To make your code work do like this:

import tensorflow as tf
from tensorflow import keras


def loss_fn(y_true, y_pred):
    return tf.nn.sigmoid_cross_entropy_with_logits(labels=y_true, logits=y_pred)

model = keras.Sequential([keras.layers.Dense(50, activation='tanh', input_shape=[100]), 
                          keras.layers.Dense(30, activation='relu'),
                          keras.layers.Dense(50, activation='tanh'),
                          keras.layers.Dense(100, activation='relu'),
                          keras.layers.Dense(8)])

model.compile(optimizer='rmsprop', 
              loss=loss_fn,
              metrics=['accuracy'])
x = np.random.normal(0, 1, (64, 100))
y = np.random.randint(0, 2, (64, 8)).astype('float32')
model.fit(x, y, epochs=10, batch_size=32)
y_pred = model.predict(x)

The suggested way, though, is to use Keras' loss implementation instead. In your case it would be:

model = keras.Sequential([keras.layers.Dense(50, activation='tanh', input_shape=[100]), 
                          keras.layers.Dense(30, activation='relu'),
                          keras.layers.Dense(50, activation='tanh'),
                          keras.layers.Dense(100, activation='relu'),
                          keras.layers.Dense(8)])

model.compile(optimizer='rmsprop', 
              loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=['accuracy'])
x = np.random.normal(0, 1, (64, 100))
y = np.random.randint(0, 2, (64, 8)).astype('float32')
model.fit(x, y, epochs=10, batch_size=32)
y_pred = model.predict(x)
user_51542740
Related