How to compute a custom loss function in R using keras with tensorflow?

Myoujin 0
Viewed 18

I am creating a custom loss function in R for Keras and Tensorflow. To make my coding attempts plausible, I tried to recreate the MSE and compare it with the inbuilt functionality of Keras. I have developed the functions my_mse() and metric_mse() in this fully reproducible example:

library(keras)
library(tensorflow)

epochs <- 3

dataset <- dataset_boston_housing()

c(c(train_data, train_targets), c(test_data, test_targets)) %<-% dataset

X_mean <- apply(train_data, 2, mean)
X_std <- apply(train_data, 2, sd)

train_data <- scale(train_data, center = X_mean, scale = X_std)
test_data <- scale(test_data, center = X_mean, scale = X_std)

set_random_seed(1234)

model <- keras_model_sequential() %>%
  layer_dense(units = 64, activation = "relu", kernel_initializer = initializer_he_uniform(),input_shape = dim(train_data)[[2]]) %>%
  layer_dense(units = 64, activation = "relu", kernel_initializer = initializer_he_uniform()) %>%
  layer_dense(units = 1)

my_mse <- function(y_true, y_pred){
  K        <- backend()
  loss <- K$mean(K$square(y_true-y_pred))
  loss
}

metric_mse <- custom_metric("my_mse", function(y_true, y_pred) {
  my_mse(y_true, y_pred)
})

model <- model %>% compile(
  loss = "mse", 
  optimizer = optimizer_adam(lr=0.001),
  metrics = metric_mse)

history <- model %>% fit(
  train_data, train_targets,
  epochs = epochs, batch_size = 2^4,
  validation_split=0.2
)

However, the results for metrics and loss are always slightly different for each epoch, e.g. "1s 33ms/step - loss: 490.4459 - my_mse: 484.7218 - val_loss: 441.9290 - val_my_mse: 446.8440". Is there something obvious I am missing?

Edit: If I take metric_mse() as loss and "mse" as metrics, then both values do actually coincide! How can that be?

Thanks in advance!

keras tensorflow r
1 Answers

The loss is not only the loss function. There may be regularizers applied on layers, they are added to the loss, too. You have to take a look on the model. That's why mse as metric may differ from the loss. If your loss function matches the mse as metric, it means your loss function works correctly.

AndrzejO0
Related