I have a sequence to sequence model designed in keras to predict numerical data but my input sequences are of varying length. The sequences are stored in a numpy array where not present values are NaNs
[[0, 1, 2, 3],
[NaN, 1, 2, NaN]]
for example. The NaNs are always on the ends if there are any, the numbers are always unbroken sequences in the data so for example the following sequence doesn't appear:
[NaN, 0, NaN, 3].
I would like to use a masking layer in my model so that these NaN values are ignored however at the moment what I have doesn't work as the loss I get is always a NaN output.
encoder_inputs = Input(shape=(None, 1), name='encoder')
masker = Masking(mask_value=np.nan)
masker(encoder_inputs)
encoder = LSTM(units, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_inputs)
# Keep encoder states for decoder, discard outputs
encoder_states = [state_h, state_c]
# Set up the decoder taking the encoder_states to be the initial state vector of the decoder.
decoder_inputs = Input(shape=(None, 1), name='decoder')
# Full output sequences and internal states are returned. Returned states are used in prediction / inference
decoder = LSTM(units, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder(decoder_inputs, initial_state=encoder_states)
# Gives continuous output at each time step
decoder_dense = Dense(1)
decoder_outputs = decoder_dense(decoder_outputs)
# create model that takes encoder_input_data and decoder_input_data and creates decoder_target_data
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
I guess I haven't correctly added the Masking layer to the model but I am not sure how to add it?