Understanding the implementation of ConvLSTM in tensorflow

user_5340080 0
Viewed 169

In the tensorflow implementation of convLSTM cell the following lines of code are written as:

    x_i = self.input_conv(inputs_i, kernel_i, bias_i, padding=self.padding)
    x_f = self.input_conv(inputs_f, kernel_f, bias_f, padding=self.padding)
    x_c = self.input_conv(inputs_c, kernel_c, bias_c, padding=self.padding)
    x_o = self.input_conv(inputs_o, kernel_o, bias_o, padding=self.padding)
    h_i = self.recurrent_conv(h_tm1_i, recurrent_kernel_i)
    h_f = self.recurrent_conv(h_tm1_f, recurrent_kernel_f)
    h_c = self.recurrent_conv(h_tm1_c, recurrent_kernel_c)
    h_o = self.recurrent_conv(h_tm1_o, recurrent_kernel_o)

    i = self.recurrent_activation(x_i + h_i)
    f = self.recurrent_activation(x_f + h_f)
    c = f * c_tm1 + i * self.activation(x_c + h_c)
    o = self.recurrent_activation(x_o + h_o)
    h = o * self.activation(c)

The corresponding equations as described in the paper are:

enter image description here

I am not able to see how W_ci, W_cf, W_co C_{t-1}, C_t is used in the input, forget and output gates. Where does it being used in computing the 4 gates?

lstm tensorflow deep-learning python
1 Answers

Of course you cannot find those in that implementation of ConvLSTM cell, because it not using peephole:

Peephole connections allow the gates to utilize the previous internal state as well as the previous hidden state (which is what LSTMCell is limited to)

The tf.keras.experimental.PeepholeLSTMCell follow the equations you post above, as you can see in it source code:

x_i, x_f, x_c, x_o = x
h_tm1_i, h_tm1_f, h_tm1_c, h_tm1_o = h_tm1
i = self.recurrent_activation(
    x_i + K.dot(h_tm1_i, self.recurrent_kernel[:, :self.units]) +
    self.input_gate_peephole_weights * c_tm1)
f = self.recurrent_activation(x_f + K.dot(
    h_tm1_f, self.recurrent_kernel[:, self.units:self.units * 2]) +
                              self.forget_gate_peephole_weights * c_tm1)
c = f * c_tm1 + i * self.activation(x_c + K.dot(
    h_tm1_c, self.recurrent_kernel[:, self.units * 2:self.units * 3]))
o = self.recurrent_activation(
    x_o + K.dot(h_tm1_o, self.recurrent_kernel[:, self.units * 3:]) +
    self.output_gate_peephole_weights * c)

Or more clear, if you look at source code of tf.compat.v1.nn.rnn_cell.LSTMCell:

if self._use_peepholes:
  c = (
      sigmoid(f + self._forget_bias + self._w_f_diag * c_prev) * c_prev +
      sigmoid(i + self._w_i_diag * c_prev) * self._activation(j))
else:
  c = (
      sigmoid(f + self._forget_bias) * c_prev +
      sigmoid(i) * self._activation(j))
user_135095400
Related