Custom loss and accuracy in Keras

user_12984908 0
Viewed 972

I want to extend the output of the network to two outputs in this code

# Model architecture
  input = Input(shape = (max_len,))
  model = Embedding(input_dim = len(words) + 2, output_dim = embedding, input_length = max_len, mask_zero = True)(input)
  model = Bidirectional(LSTM(units = 50, return_sequences=True, recurrent_dropout=0.1))(model)
  model = TimeDistributed(Dense(50, activation="relu"))(model)
  crf = CRF(num_tags+1)  # CRF layer
  out = crf(model)  # output

  model = Model(input, out)
  model.compile(optimizer="rmsprop", loss=crf.loss_function, metrics=[crf.accuracy])

(The original code is at https://github.com/Akshayc1/named-entity-recognition.git). I want the loss function to be the summation of losses for these two outputs, and accuracy to be mean of accuracy for these two outputs, I followed the available instructions:

model = TimeDistributed(Dense(50, activation="relu"))(model)
model1 = TimeDistributed(Dense(50, activation="relu"))(model)
crf1 = CRF(num_tags+1)  # CRF layer
crf2 = CRF(num_tags+1)
out1 = crf1(model)  # output
out2 = crf2(model1) 

model = Model(input, [out1, out2])

def custom_loss(y_true, y_pred1,  y_pred2):
    loss1 = losses.categorical_crossentropy(y_true,y_pred1)
    loss2 = losses.categorical_crossentropy(y_true,y_pred2)
    return (loss1 + loss2)/2


def Custom_accuracy(y_true, y_pred1,y_pred2, k=10):
    acc1 = K.mean(K.in_top_k(y_pred, K.argmax(y_true, axis=-1), k), axis=-1)
    acc2 = K.mean(K.in_top_k(y_pred1, K.argmax(y_true, axis=-1), k), axis=-1)
    acc = (acc1 + acc2)/2
    return acc
model.compile(optimizer="rmsprop", loss=custom_loss,metrics=Custom_accuracy)

but this shows the error:

TypeError: custom_loss() missing 1 required positional argument: 'y_pred2'
keras python
2 Answers

It can be solved by passing two loss functions to loss argument in model.compile than to pass three variables in loss function as described in the documentation and also make classes for custom metric and loss. Make the following changes - 

...

crf1 = CRF(num_tags+1,name="out1") <-- # change 1
crf2 = CRF(num_tags+1,name="out2") <-- # change 2
out1 = crf1(model)  
out2 = crf2(model1) 

model = Model(input, [out1, out2])

<define accuracy class and create its object> <-- # change 3
<define loss class and create its object> <-- # change 4

model.compile(optimizer="rmsprop", 
            loss={"out1":<loss_object_1>,"out2":<loss_object_2>},
            metrics={"out1":<accuracy_object_1>,"out2":<accuracy_object_2>}) <-- # change 5
user_123334550

I think that is because the output of the model is not two outputs values( like o1,o2), but a list of two values(like [o1,o2]). Try this custom loss function, hopefully it gets the issue gone:

def custom_loss(y_true, y_predsList):
    y_pred1,  y_pred2 = y_predsList
    loss1 = losses.categorical_crossentropy(y_true,y_pred1)
    loss2 = losses.categorical_crossentropy(y_true,y_pred2)
    return (loss1 + loss2)/2
user_110653470
Related