My model layers are using relu activation function. I am using he_uniform for the kernel initializer, but i saw kaiming initialization giving better result than he_uniform. I m using keras, and keras has no kaiming initializer, how can I implement it?
1 Answers
The confusion arise from the naming convention, the difference is naming with first name or surname. Basically, the initialization method proposed in this paper co-authored by Kaiming He. The framework implementations differ, however;
Tensorflow via Keras backend uses the name He initilization. Torch uses Kaiming initilization as the method names.
In fact, the same applies also for Glorot/Xavier initialization. See the paper co-authored by Xavier Glorot.
Here, Tensorflow via Keras uses the surname Glorot whereas Torch uses the first name Xavier.
Related