Correct way of adding a masking layer to a GRU model using the functional API in Keras?

user_4680773 0
Viewed 398

I am trying to figure out how to use a Masking layer with the functional API in Keras.

Using the non-functional Keras syntax I can easily create a GRU model that masks away all zero values like this:

model = tf.keras.Sequential()
model.add( tf.keras.layers.Masking( mask_value = 0.0, input_shape = ( nTimeSteps, nVariables ) ) )
model.add( tf.keras.layers.GRU( 32 ) )
model.add( tf.keras.layers.Dense( 10, activation = "softmax" ) )
opt = tf.keras.optimizers.SGD( learning_rate = 0.001 )
model.compile( loss = 'categorical_crossentropy', optimizer = opt, metrics = ['accuracy'] )

My attempt at replicating this model using the functional API so far looks like this:

x = tf.keras.layers.Masking( mask_value = 0.0, input_shape = ( nTimeSteps, nVariables ) )
x = tf.keras.layers.GRU( 32 )( x )
z = tf.keras.layers.Dense( numberOfOutputs, activation = "softmax" )( x )
model = tf.keras.Model( inputs = x, outputs = z )
opt = tf.keras.optimizers.SGD( learning_rate = 0.001 )
model.compile( loss = 'categorical_crossentropy', optimizer = opt, metrics = ['accuracy'] )

However, it does not work – it produces the following error:

AttributeError: 'Masking' object has no attribute 'shape'

What is the correct way of using a masking layer with the functional API?

keras tensorflow deep-learning machine-learning python
1 Answers

you miss the Input layer in the functional API format. Here a dummy example

nsamples = 10
nTimeSteps, nVariables = 6, 4
numberOfOutputs = 2
X = np.random.randint(0,6, (nsamples ,nTimeSteps, nVariables))
y = np.random.randint(0,numberOfOutputs, nsamples)

inp = tf.keras.Input(shape = ( nTimeSteps, nVariables ))
x = tf.keras.layers.Masking( mask_value = 0.0 )(inp)
x = tf.keras.layers.GRU( 32 )( x )
z = tf.keras.layers.Dense( numberOfOutputs, activation = "softmax" )( x )

model = tf.keras.Model( inputs = inp, outputs = z )
opt = tf.keras.optimizers.SGD( learning_rate = 0.001 )
model.compile( loss = 'sparse_categorical_crossentropy', 
              optimizer = opt, metrics = ['accuracy'] )

model.fit(X,y, epochs=3)
user_103750490
Related