Merging LSTM and 1D CNN layers

user_7020892 0
Viewed 698

I'm trying to test Non-Intrusive Load Monitoring algorithm which was Proposed by Kelly, 2015. The proposed architecture was,

  1. Input (length determined by appliance duration)
  2. 1D conv (lter size=4, stride=1, number of lters=16,activation function=linear, border mode=same)
  3. bidirectional LSTM (N=128, with peepholes)
  4. bidirectional LSTM (N=256, with peepholes)
  5. Fully connected (N=128, activation function=TanH)
  6. Fully connected (N=1, activation function=linear)

Now I'm going to test this architecture in TensorFlow. I have an initial training set like fallows,

x_train_temp = (243127,) y_train_temp = (243127,)

Then I converted these data into windows like below where each window has 250 x1 arrays.

x_train = (972, 250, 1) y_train = (972, 250, 1)

When I'm implementing my model, It gives an error. Could you please help me to understand where the error is?

Model

input_shape=x_train.shape[1:]
model = Sequential()
model.add(tf.keras.layers.Conv1D(16,4, strides=1,activation='linear',input_shape=(x_train.shape[1:])))
model.add(tf.keras.layers.LSTM(128))
model.add(tf.keras.layers.LSTM(256))
model.add(tf.keras.layers.LSTM(128, activation='relu'))
model.add(tf.keras.layers.LSTM(128, activation='linear'))
print(model.summary())

Error

ValueError                                Traceback (most recent call last)
<ipython-input-47-4f5e2441909e> in <module>()
      4 model.add(tf.keras.layers.Conv1D(16,4, strides=1,activation='linear',input_shape=(x_train.shape[1:])))
      5 model.add(tf.keras.layers.LSTM(128))
----> 6 model.add(tf.keras.layers.LSTM(256))
      7 model.add(tf.keras.layers.LSTM(128, activation='relu'))
      8 model.add(tf.keras.layers.LSTM(128, activation='linear'))

5 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
    178                          'expected ndim=' + str(spec.ndim) + ', found ndim=' +
    179                          str(ndim) + '. Full shape received: ' +
--> 180                          str(x.shape.as_list()))
    181     if spec.max_ndim is not None:
    182       ndim = x.shape.ndims

ValueError: Input 0 of layer lstm_1 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 128]
recurrent-neural-network lstm keras tensorflow python
1 Answers

When using stacked LSTM layers use return_sequences=True as mentioned by @Ather Cheema. LSTM expects input of shape 3D tensor with shape [batch, timesteps, feature]. It is highly recommended to set return_sequences=True

return_sequences: Boolean. Whether to return the last output. in the output sequence, or the full sequence. Default: False.

Working sample code snippet Without retuen_sequences

import tensorflow as tf
inputs = tf.random.normal([32, 10, 8])
lstm = tf.keras.layers.LSTM(4)
output = lstm(inputs)
print(output.shape)

Output

(32, 4)

#With return_sequences=True

import tensorflow as tf
inputs = tf.random.normal([32, 10, 8])
lstm = tf.keras.layers.LSTM(4, return_sequences=True, return_state=True)
whole_seq_output, final_memory_state, final_carry_state = lstm(inputs)
print(whole_seq_output.shape)

Output

(32, 10, 4)
user_142902440
Related