Passing a Custom mask to the LSTM data for training and validation

user_13469674 0
Viewed 167

I have an LSTM architecture ready:

input1 = Input(shape=(1500, 3))
lstm = LSTM(units=100, return_sequences=False, activation='relu')(input1)
outputs = Dense(150, activation="sigmoid")(lstm)
model = Model(inputs=input1, outputs=outputs)
model.compile(loss="binary_crossentropy", optimizer="adam",
                  metrics=["accuracy"])

The LSTM layer supports a calling argument called mask.

The way I'm reading the data is by using two generators, one iterates through training files and the other through the validation files (so on the .fit method I pass the training and validation generators).

model.fit(
        x=training_generator,
        epochs=10,
        steps_per_epoch=5, # there are 5 files
        validation_data=validation_generator,
        validation_steps=5, # there are 5 files
        verbose=1
    )

Therefore each file will have a given mask (one for the training file, another for the validation file). Therefore my question is, how can I specify which mask to use?

lstm keras tensorflow python
1 Answers

The way I found to work was to transform the data during the preprocessing stage. If you replace the values in your data, according to the mask, with an number you know is not in your data, for instance 0 or -999, you can then add another layer to the architecture called Masking. This layer has a parameter called mask_value which will be the same number you used to transform your data:

input1 = Input(shape=(n_timesteps, n_channels))
masking = Masking(mask_value=-999)(input1)
lstm1 = LSTM(units=100, return_sequences=False,
             activation="tanh")(masking)
outputs = Dense(n_timesteps, activation="sigmoid")(lstm1)
model = Model(inputs=input1, outputs=outputs)
model.compile(loss=keras.losses.BinaryCrossentropy(),
              optimizer=tf.keras.optimizers.Adam(learning_rate=0.01))

This way you can then pass this as the input to the LSTM (since LSTMs allow this, some other types of layers do not).

user_134696740
Related