To improve performance of my neural network i'm trying to use the learning rate decay technique, but I can't understand how it works. Some tutorial says that the decay value can be fixed (but I have not any idea of the right value) and others say that should be a formula. Possible example of formula could be decay=actual learning rate/actual epoch But I'm using R and I don't know how to access to the number of epoch
this is the configuration now:
# Network config
history <- model %>% compile(
loss = 'binary_crossentropy',
optimizer = optimizer_sgd(
learning_rate=0.1,
momentum = 0.0,
decay = 0.0,
nesterov = FALSE
) ,
metrics = c('accuracy')
)
the dataset is quite small with 32 col and 569 row, all numeric values.
Any ideas/example?