What is the difference between clipnorm and clipval on Keras

user_4752738 0
Viewed 4257

What is the difference between clipnorm and clipval.

Ex: opt = SGD(lr=0.01, momentum=0.9, clipnorm=1.0)

gradient-descent keras tensorflow deep-learning optimization
1 Answers

Incase of clipnorm, the l2 norm of the gradients is capped at the specified value.

While clipvalue caps the gradient values such that they don't exceed the specified value

user_95790750
Related