Keras LSTM parameter explanation

user_9142291 0
Viewed 318

I have this neural network model to create anomaly detection model. I copy this model from one of a tutorial website

def autoencoder_model(X):
  inputs = Input(shape=(X.shape[1], X.shape[2]))
  L1 = LSTM(16, activation='relu', return_sequences=True, kernel_regularizer = regularizers.l2(0.00))(inputs)
  L2 = LSTM(4, activation = 'relu', return_sequences=False)(L1)
  L3 = RepeatVector(X.shape[1])(L2)
  L4 = LSTM(4, activation='relu', return_sequences=True)(L3)
  L5 = LSTM(16, activation = 'relu', return_sequences=True)(L4)
  output = TimeDistributed(Dense(X.shape[2]))(L5)
  model = Model(inputs=inputs, outputs=output)
  return model

I know that 16 or 4 is number of perceptron. but I don't know the return_sequences means and why L2 and L3 have to be False. And I don't know what kernel_regularizer means. I already read the documentation but still I don't understand

lstm keras tensorflow neural-network python
1 Answers

You are correct, 16, 4... are number of LSTM cells. About return sequences, here is need to understand what is LSTM input. LSTM input have shape time steps, features(I not assume here batch dimension).

Maybe example will be better for explanation, let say you want to predict average temperature for next hour based on past few hours and humidity. So your data looks like(just concept, no real deal values)

[[32, 10],
[27, 12],
[26, 10],
[25, 11],
[27, 10]]

So we have humidity and temperature for 5 hours. We gave this chunk to LSTM layer and he process it. LSTM works sequentionaly so it take [32, 10] do computation and gave some result. LSTM gave result for every temperature humidty pair so if layer has 4 cells for our example we expect output 5 x 4(because we have 5 pairs and 4 cells). What return sequences do is gave you option say LSTM I don't care about response to every pair, just gave me some final result, so if you set it as False(default value), than instead of 5 x 4 output you get 1 x 4 output(just final value for processing of all 5 pairs).

About regularizer I think it is out of the scope for this Q & A format. So lets say that regularizer add term to gradient computation according to loss function in order to avoid overfitting. I also point you to this(I believe it is pretty good as introduction).

user_67063560
Related