The difference between loss in model.compile() and model.add_loss() in Keras

user_11673522 0
Viewed 1352

I am puzzled about what is difference between the add_loss and the traditional loss in model.compile()??

My code is like following:

from time import time
import numpy as np
import random
from keras.models import Model
import keras.backend as K
from keras.engine.topology import Layer, InputSpec
from keras.layers import Dense, Input, GaussianNoise, Layer, Activation
from keras.models import Model
from keras.optimizers import SGD, Adam
from keras.utils.vis_utils import plot_model
from keras.callbacks import EarlyStopping


input_place = Input(shape=(128,))

e_layer1 = Dense(64,activation='relu')(input_place)
e_layer2 = Dense(32,activation='relu')(e_layer1)
hidden = Dense(16,activation='relu')(e_layer2)

d_layer1 = Dense(32,activation='relu')(hidden)
d_layer2 = Dense(64,activation='relu')(d_layer1)

output_place = Dense(128,activation='sigmoid')(d_layer2)

model = Model(inputs=input_place,outputs=output_place)

loss = K.mean(K.square(d_layer1 - e_layer2),axis = -1)

model.add_loss(loss)

model.compile(optimizer = 'adam',
              loss=['mse'],
              metrics=['accuracy'])

input_data = np.random.randn(1,128)

model.fit(input_data,
          input_data,
          epochs=5)

As mentioned above, I made two loss function, one is a traditional MSE loss in model.compile() to compute the MSE_loss of input and output, and the other loss is also like a MSE loss, but it computes the middle_layers' MSE. It can run, but I puzzled, with these two different ways to add loss, can my model know what they are clearly??

loss keras tensorflow python
1 Answers

Yes, your Model knows what they are, indeed.

The loss specified in the model.compile ensures that the MSE between Y_Pred and Y_Actual is reduced and the model.add_loss ensures that the difference between d_layer1 and e_layer2 is reduced.

It is equivalent to specifying two losses inside model.compile but the basic difference between the loss specified in model.compile and model.add_loss is that the loss specified in model.compile is restricted to the arguments, y_true and y_pred whereas in model.add_loss, we can specify the Loss with respect to any number of Additional Tensors used in our Project.

In other words, model.add_loss allows us to write much more complex losses that depend on many other tensors, but it has the inconvenience of being more dependent on the model, whereas the standard loss functions (those used inside model.compile) work with just any model.

Hope this helps. Happy Learning!

user_134652580
Related