So I have been working with a sequential Keras model in Tensor-Flow, and have come across an odd behavior where if I compile a sequential model with the loss as a function, the result is different than if it were a string.
The network (convolutional in nature) is defined as such:
model = tf.keras.Sequential()
add_model_layers(model) # Can provide if needed
adam_opt = tf.train.AdamOptimizer(learning_rate=0.001,
beta1=0.9,
beta2=0.999)
The network is then compiled:
loss_param1 = tf.keras.losses.categorical_crossentropy
loss_param2 = "categorical_crossentropy"
model.compile(optimizer=adam_opt,
loss=loss_param # ADD NUMBER TO END
metrics=[tf.keras.metrics.categorical_accuracy])
After which it is trained:
# Records are tf.data.Dataset based on TFRecord files
model.fit(train_records,
epochs=400,
use_multiprocessing=False,
validation_data=validation_records)
# And for completeness, tested
model.evaluate(test_records)
If the loss parameter of model.compile is loss_param1, it begins training with a high loss value (in my case, after an epoch or 5 around 192). On the other hand, if loss_param2 is used, training begins at a much lower loss (around 41).
Does anyone know why this would be occurring?
(As an additional note, I am also running into a similar issue where if the metric is as a string I get a different result. However in model.fit, if use_multiprocessing is True the effect is negated (this also applies the other way around).)