I have 1-dimensional signal which I would like to translate from integer electrical current to a DNA sequence (4-word alphabet) using seq2seq.
My input data is of the shape [batch_size, 1000, 1] and my output data [batch_size, ? (~100), embedding_size] where I encode each word in my base alphabet to a fixed-length vector of floats (between 4 and 64, longer seems to be better). I send this through a 2-layer, 64-unit LSTM encoder, and then apply Bahdanau Attention on a 2-layer, 64-unit LSTM decoder. Unfortunately, most of my outputs seem to be approximately 5 words in length.
I have read that seq2seq can handle "long" sentences (Sutskever et al., page 7) but as yet I am unable to replicate this success on what is a very small output vocabulary. Ideally I would like to increase my input length by a factor of 10 or 100 in the long run, if this is possible.
I am concerned that perhaps the encoded 'thought vector' simply is not storing enough information for the long sequence. Does anyone have experience / ideas on how to make a Seq2Seq model work with long sentences?