Using an Embedding layer in the Keras functional API

user_5968729 0
Viewed 1059

Doing basic things with the Keras Functional API seems to produce errors. For example, the following fails:

from keras.layers import InputLayer, Embedding
input = InputLayer(name="input", input_shape=(1, ))
embedding = Embedding(10000, 64)(input)

This produces the error:

AttributeError: 'str' object has no attribute 'base_dtype'

I can then "cheat" by using the input_length argument but this then fails when I try to concatenate two such embeddings:

from keras.layers import InputLayer, Embedding, Concatenate
embedding1 = Embedding(10000, 64, input_length=1)
embedding2 = Embedding(10000, 64, input_length=1)
concat = Concatenate()([embedding1 , embedding2])

This gives the error:

TypeError: 'NoneType' object is not subscriptable

Same error when I use "concatenate" (lower case) instead (some sources seem to say that this should be used instead if using the functional API).

What am I doing wrong?

I am on tensorflow version 2.3.1, keras version 2.4.3, python version 3.6.7

keras tensorflow python-3.x
1 Answers

I strongly suggest to use tf.keras and not keras.

It doesn't work because InputLayer is an instance of keras.Layer, whereas keras.layers.Input is an instance of Tensor. The argument to layer.__call__() should be Tensor and not keras.Layer.

import tensorflow as tf

inputs = tf.keras.layers.Input((1,))
print(type(inputs)) # <class 'tensorflow.python.framework.ops.Tensor'>
input_layer = tf.keras.layers.InputLayer(input_shape=(1,))
print(type(input_layer)) # <class 'tensorflow.python.keras.engine.input_layer.InputLayer'>

You use InputLayer with Sequential API. When you use functional API you should use tf.keras.layers.Input() instead:

import tensorflow as tf

inputs = tf.keras.layers.Input((1, ), name="input", )
embedding = tf.keras.layers.Embedding(10000, 64)(inputs)

Same with the second example:

import tensorflow as tf

inputs = tf.keras.layers.Input((1, ), name="input", )
embedding1 = tf.keras.layers.Embedding(10000, 64)(inputs)
embedding2 = tf.keras.layers.Embedding(10000, 64)(inputs)

concat = tf.keras.layers.Concatenate()([embedding1, embedding2])
user_51542740
Related