Keras: the difference between LSTM dropout and LSTM recurrent dropout

user_1998149 0
Viewed 38702

From the Keras documentation:

dropout: Float between 0 and 1. Fraction of the units to drop for the linear transformation of the inputs.

recurrent_dropout: Float between 0 and 1. Fraction of the units to drop for the linear transformation of the recurrent state.

Can anyone point to where on the image below each dropout happens?

enter image description here

dropout lstm keras
2 Answers

Above answer highlights one of the recurrent dropout methods but that one is NOT used by tensorflow and keras. Tensorflow Doc.

Keras/TF refers a recurrent method proposed by Semeniuta et al. Also, check below the image comparing different recurrent dropout methods. The Gal and Ghahramani method which is mentioned in above answer is at second position and Semeniuta method is the right most.

enter image description here

user_60256140
Related