'LayerNormLSTMCell' object has no attribute 'zero_state' in tf 2.2

user_13961251 0
Viewed 366

My initial code (tf 1.12) is below

def _build_single_cell(self, lstm_units, keep_prob):
    cell = tf.contrib.rnn.LayerNormBasicLSTMCell(lstm_units)
    cell = tf.contrib.rnn.DropoutWrapper(cell, input_keep_prob=keep_prob, output_keep_prob=keep_prob)

    return cell

I changed it to tf 2.2 code. Like this

def _build_single_cell(self, lstm_units, keep_prob):
    cell = tfa.rnn.LayerNormLSTMCell(lstm_units)
    cell = tf.compat.v1.nn.rnn_cell.DropoutWrapper(cell, input_keep_prob=keep_prob, output_keep_prob=keep_prob)

    return cell

But error like this happened

AttributeError: 'LayerNormLSTMCell' object has no attribute 'zero_state'

How can I change it?

tensorflow
1 Answers

I will suggest to remove the DropoutWrapper layer and use directly the Droupout parameter that is available in tfa.rnn.LayerNormLSTMCell. From the API

tfa.rnn.LayerNormLSTMCell(
    units: tfa.image.color_ops.TensorLike,
    activation: tfa.rnn.cell.Activation = 'tanh',
    recurrent_activation: tfa.rnn.cell.Activation = 'sigmoid',
    use_bias: bool = True,
    kernel_initializer: tfa.rnn.cell.Constraint = 'glorot_uniform',
    recurrent_initializer: tfa.rnn.cell.Constraint = 'orthogonal',
    bias_initializer: tfa.rnn.cell.Constraint = 'zeros',
    unit_forget_bias: bool = True,
    kernel_regularizer: tfa.rnn.cell.Constraint = None,
    recurrent_regularizer: tfa.rnn.cell.Constraint = None,
    bias_regularizer: tfa.rnn.cell.Constraint = None,
    kernel_constraint: tfa.rnn.cell.Constraint = None,
    recurrent_constraint: tfa.rnn.cell.Constraint = None,
    bias_constraint: tfa.rnn.cell.Constraint = None,
    dropout: tfa.image.filters.FloatTensorLike = 0.0,
    recurrent_dropout: tfa.image.filters.FloatTensorLike = 0.0,
    norm_gamma_initializer: tfa.rnn.cell.Constraint = 'ones',
    norm_beta_initializer: tfa.rnn.cell.Constraint = 'zeros',
    norm_epsilon: tfa.image.filters.FloatTensorLike = 0.001,
    **kwargs
)

If you follow the Github issue, it is suggested to use the Droupout parameter in place of using the DropuoutWrapper as that causes exactly the same issue as your's.

user_72205450
Related