Convex Combination between two Categorical cross entropy

user_13714390 0
Viewed 86

I am trying to compute a custom loss function in Keras using Categorical Cross entropy. I would like to create a unique loss function for both outputs (my network has 1 input and 2 outputs) that is:

L= lambda*L1+(1-lambda)*L2

where lambda is between 0 and 1 and L1 is the categorical loss entropy of the first output and L2 of the second... I tried like this:

def my_loss(y_true, y_pred):
    final_loss = (0.8*(losses.binary_crossentropy(y_true[:, 0], y_pred[:, 0])+(0.2)*( losses.categorical_crossentropy(y_true[:, 1:], y_pred[:,1:]))))
    return final_loss

and then:

 model.compile(optimizer='Adam', loss=[my_loss],metrics=[metrics.categorical_accuracy])

But first problem is that i have still pass 2 times the function my loss and I don't know if is it correct; plus I can`t pass lambda to my_loss.

How can I do?

loss-function keras deep-learning python
1 Answers

I think that all that you need is available in keras

inp = Input((10))
x = Dense(32)(inp)
out1 = Dense(1, activation='sigmoid', name='out1')(x)
out2 = Dense(1, activation='sigmoid', name='out2')(x)

m = Model(inp, [out1, out2])
alpha = 0.3
m.compile('adam', loss={'out1':'binary_crossentropy', 'out2':'binary_crossentropy'}, 
          loss_weights={'out1':alpha,'out2':1-alpha})

X = np.random.uniform(0,1, (1000,10))
y1 = np.random.uniform(0,1, 1000)
y2 = np.random.uniform(0,1, 1000)

m.fit(X, [y1,y2], epochs=10)

you can define a model with multiple outputs and multiple losses

the final loss is the weighted average loss between the loss you provided... loss = out1_loss * alpha + out2_loss * (1-alpha)

user_103750490
Related