I have a collection of sequences. I would like to encode them as vectors of length encoded_dim. To that end, I made an autoencoder using stacked LSTMs in Keras:
inputs = Input(shape=(None, input_dim))
encoded = LSTM(encoded_dim)(LSTM(latent_dim, return_sequences=True)(inputs))
decoded = RepeatVector(timesteps)(encoded)
decoded = Dense(8)(Dense(100)(LSTM(input_dim, return_sequences=True)(decoded)))
sequence_autoencoder = Model(inputs, decoded)
sequence_autoencoder.compile(optimizers.Adam(lr=0.01), loss='mean_squared_error')
sequence_autoencoder.fit(input, input, epochs=3000, batch_size=16, shuffle=False, callbacks=[...], validation_split=.2)
This works well when I restrict my attention to sequences of length timesteps. In general, however, my sequences are not the same length, so RepeatVector(timesteps) won't work as is. (This will also mess up batching, but I have had decent training with batch_size=1 so I'm not too worried about that for my particular use case.)
There is a fruitful conversation here involving variable length input sequences and batching, in the context of general seq2seq modeling. Unfortunately, there is no concern there for retrieving the fixed size encoding.
Any suggestions/code snippets are greatly appreciated.