Backward propagation in Keras?

user_7855502 0
Viewed 25856

can anyone tell me how is backpropagation done in Keras? I read that it is really easy in Torch and complex in Caffe, but I can't find anything about doing it with Keras. I am implementing my own layers in Keras (A very beginner) and would like to know how to do the backward propagation.

Thank you in advance

keras-layer keras deep-learning python
1 Answers

You simply don't. (Late edit: except when you are creating custom training loops, only for advanced uses)

Keras does backpropagation automatically. There's absolutely nothing you need to do for that except for training the model with one of the fit methods.

You just need to take care of a few things:

  • The vars you want to be updated with backpropagation (that means: the weights), must be defined in the custom layer with the self.add_weight() method inside the build method. See writing your own keras layers.
  • All calculations you're doing must use basic operators such as +, -, *, / or backend functions. By backend, tensorflow/theano/CNTK functions are also supported.
  • The functions you use must be differentiable (that means backpropagation will fail for functions that use constant results, for instance)

This is all you need to have the automatic backpropagation working properly.

If your layers don't have trainable weights, you don't need custom layers, create Lambda layers instead (only calculations, no trainable weights).

user_20972400
Related