Can we use multiple loss functions in same layer?

user_12924273 0
Viewed 658

Can we use mulitple loss function in this architecture: I have two different type of loss functions and want to use it on last layer [Output] loss functions :

  • binary_crossentropy
  • custom loss function

Can we do that? enter image description here

loss-function keras tensorflow deep-learning machine-learning
2 Answers

yes you can... you simply have to repeat 2 times the model output in the model definition. you can also merge your loss in a different way using loss_weights params (default is [1,1] for two losses). Below an example in a dummy regression problem. https://colab.research.google.com/drive/1SVHC6RuHgNNe5Qj6IOtmBD5geAJ-G9-v?usp=sharing

def rmse(y_true, y_pred):

    error = y_true-y_pred

    return K.sqrt(K.mean(K.square(error)))


X1 = np.random.uniform(0,1, (1000,10))
X2 = np.random.uniform(0,1, (1000,10))
y = np.random.uniform(0,1, 1000)

inp1 = Input((10,))
inp2 = Input((10,))
x = Concatenate()([inp1,inp2])
x = Dense(32, activation='relu')(x)
out = Dense(1)(x)

m = Model([inp1,inp2], [out,out])
m.compile(loss=[rmse,'mse'], optimizer='adam') # , loss_weights=[0.3, 0.7]
history = m.fit([X1,X2], [y,y], epochs=10, verbose=2)
user_103750490

You can calculate two different losses. Then get weighted mean and return as the final value of the loss. Technically, it can be implemented like this (that is an example, I didn't run it):

def joint_loss(y_true, y_pred):
    part_binary_crossentropy = 0.4
    part_custom = 0.6

    # binary_crossentropy
    loss_binary_crossentropy = tf.keras.losses.binary_crossentropy(y_true, y_pred)

    # custom_loss
    loss_custom = some_custom_loss(y_true, y_pred))

    return part_binary_crossentropy * loss_binary_crossentropy + part_custom * loss_custom


model.compile(loss=joint_loss, optimizer='Adam')
user_86093300
Related