How to verify if masking is appropriately applied? LSTM performance drop with short sequences

Viewed 100

I am trying to understand how to apply masking and make sure that my output is masked. I use the following model to mask the inputs:

def tdcnn2d_bilstm_mask():
    with tf.device('/gpu:2'):
        inputlayer1 = Input(shape = input_shape1)
        x = TimeDistributed(Conv2D(4, kernel_size=(3, 3), padding='same', activation='relu'))(inputlayer1)
        x = TimeDistributed(MaxPooling2D(pool_size=(2,2)))(x)
        x = TimeDistributed(Conv2D(8, kernel_size=(3, 3), padding='same', activation='relu'))(x)
        x = TimeDistributed(MaxPooling2D(pool_size=(2,2)))(x)
        x = TimeDistributed(Conv2D(8, kernel_size=(3, 3), padding='same', activation='relu'))(x)
        x = TimeDistributed(MaxPooling2D(pool_size=(2,2)))(x)
        x = TimeDistributed(Conv2D(8, kernel_size=(3, 3), padding='same', activation='relu'))(x)
        x = TimeDistributed(MaxPooling2D(pool_size=(2,2)))(x)
        x = TimeDistributed(Flatten())(x)
        mask = Masking(mask_value=0.)(x)
        x = Bidirectional(LSTM(64, dropout=0.5, return_sequences=True))(mask)
        out1 = TimeDistributed(Dense(num_class,activation='softmax'))(x)
        model = keras.Model(inputs = [inputlayer1], outputs = [out1])#, out2, out3])
        opt = Adam(lr=1e-3, decay=1e-3 / 200)
        model.compile(loss = 'categorical_crossentropy', optimizer=opt,metrics = ['accuracy'])
        return model

My questions are:

  • How can I check if my input is appropriately masked by LSTM?
  • Does softmax layer output predictions for masked values?
  • How can I compare masked vs non-masked outputs?

One last unrelated question is:

  • My testing performance drops a lot when I divide test sequences to shorter sequences and try to predict them. What might be the reason for that? For example when I test with whole sequence of data (30 sec), I get 93% testing accuracy, but when I divide my test data into 0.2 sec chunks and try to predict them, accuracy drops down to around 75%. How can I overcome this problem? Thanks in advance!

Best,

Emre

0 Answers
Related