Creating a two layer Simple RNN model with Python and Keras

user_11137413 0
Viewed 441

I'm trying to create a model for a two layer SimpleRNN model using Keras library in Python. I don't really understand how the keras model layers work, so I don't quite understand how to get the result I want or how to fix the error I'm receiving.

The first SimpleRNN layer needs to be 64 dimensions and output the state sequence, whereas the second one has to have the 64 dimensions and output the final state.

I currently have:

 import tensorflow.keras.models as models
 import tensorflow.keras.layers as layers
    
 model_simple_rnn = models.Sequential([
        layers.Embedding(100,64, name='embedding'),
        layers.SimpleRNN(64, name='simpleRNN1'),
        layers.SimpleRNN(64, name='simpleRNN2'),
        layers.Dense(1, activation='sigmoid', name='dense'),
        ], name='SIMPLERNN')

I'm getting the following error:

ValueError: Input 0 of layer simple_rnn_1 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 64)

I need to have the following output:

Model: "SIMPLERNN"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding (Embedding)        (None, None, 100)         1000000   
_________________________________________________________________
simpleRNN1 (SimpleRNN)       (None, None, 64)          10560     
_________________________________________________________________
simpleRNN2 (SimpleRNN)     (None, 64)                8256      
_________________________________________________________________
dense (Dense)                (None, 1)                 65        
=================================================================
Total params: 1,018,881
Trainable params: 1,018,881
Non-trainable params: 0
keras machine-learning python
2 Answers
 import tensorflow.keras.models as models
 import tensorflow.keras.layers as layers
    
 model_simple_rnn = models.Sequential([
        layers.Embedding(100,64, name='embedding'),
        layers.SimpleRNN(return_sequences=True, 64, name='simpleRNN1'),
        layers.SimpleRNN(64, name='simpleRNN2'),
        layers.Dense(1, activation='sigmoid', name='dense'),
        ], name='SIMPLERNN')

Try this code. It may help.

If it still doesn't work, please mention the input size in the first layer properly. It can be one of the reasons too if not working.

user_98480430

I think you need to specify one more argument in the SimpleRNN function. Namely, return_sequences=True, in the first of the two SimpleRNN layers, and then return_sequences=False in the second.

Basically, True means many-to-many, which is what you want when feeding it to the next RNN layer, and False is many-to-one needed for the Dense layer.

user_178849050
Related