How to set learning rate decay with keras and tensorflow in R

Viewed 21

To improve performance of my neural network i'm trying to use the learning rate decay technique, but I can't understand how it works. Some tutorial says that the decay value can be fixed (but I have not any idea of the right value) and others say that should be a formula. Possible example of formula could be decay=actual learning rate/actual epoch But I'm using R and I don't know how to access to the number of epoch

this is the configuration now:

# Network config
history <- model %>% compile(
  loss = 'binary_crossentropy',
  optimizer = optimizer_sgd( 
    learning_rate=0.1, 
    momentum = 0.0, 
    decay = 0.0, 
    nesterov = FALSE
  ) ,
  metrics = c('accuracy')
)

the dataset is quite small with 32 col and 569 row, all numeric values.

Any ideas/example?

0 Answers
Related