I would like to change the following code from tf1.0 to tf2.0
tf.contrib.layers.layer_norm(
inputs=input_tensor, begin_norm_axis=-1, begin_params_axis=-1, scope=name)
This code is taken from https://github.com/google-research/bert/blob/master/modeling.py Line 364.
Please help me.
@rishabh-sahrawat's answer is right, but you should do something like this:
layer_norma = tf.keras.layers.LayerNormalization(axis = -1)
layer_norma(input_tensor)
In the BERT case you linked, you should modify the code with something like this:
def layer_norm(input_tensor, name=None):
"""Run layer normalization on the last dimension of the tensor."""
layer_norma = tf.keras.layers.LayerNormalization(axis = -1)
return layer_norma(input_tensor)
This is the equivalent way to do this in TF2.0
tf.keras.layers.LayerNormalization(input_tensor, axis = -1)