Error in Keras' lambda layer while adding or multiplying with constant non-tensor numbers

user_11693382 0
Viewed 710

I try to custom-design a network architecture that I can have a layer as follows: 

x = k.Input(shape=(1,))
y = k.layers.Dense(1)(x + 1)  #k.backend.constant(1) -- no difference
Fx = k.models.Model(x, y)

While "x+1" is a correct Tensorflow operation, however, I get the NoneType error:

AttributeError: 'NoneType' object has no attribute '_inbound_nodes'

When I try to use the Lambda layer to circumvent this situation, I get the same error:

x = k.Input(shape=(1,))
xx = k.layers.Lambda(lambda x: x[0] + x[1])(
    [x, k.backend.constant(1, shape=(1,1))]
)
y = k.layers.Dense(1)(xx)
Fx = k.models.Model(x, y)

I however can hack through this by doing: 

x = k.Input(shape=(1,))
xx = k.layers.Lambda(lambda x: x[0]+1)([x, x])
y = k.layers.Dense(1)(xx)
Fx = k.models.Model(x, y)

Because Tensor+(int or float) is a legitimate Tensorflow operation.

Am I making a mistake in defining the Lambda layer or this is a bug on Keras end?

keras tensorflow layer lambda python
2 Answers

The reason that y = k.layers.Dense(1)(x + 1) is incorrect is that + 1 is a valid tensor operation. Keras models as defined in terms of Keras layers; not tensors. So you need to use x_p = Lambda(lambda x: x + 1)(x). The later is the correct way to specify the operation you want.

i.e. lambda x: x + 1 is the operation on tensors while Lambda() instantiates a layer. Layers is how Keras keeps track of dependencies between layers / graph construction, etc. For instance that lambda layer is going to know the format of its input shape when Keras builds the graph and then calculate the output shape. In your case the shapes are the similar but for instance you can use a lambda function that performs a tensor slicing operation and change the output shape.

user_52989370

I don't know why you change from 

x + 1

to 

 x[0]+1

if you just want to add 1 to the elements in x before passing it through a dense layer then you can just apply a lambda layer like this:

x = k.Input(shape=(1,))
xx = k.layers.Lambda(lambda x: x + 1)(x)
y = k.layers.Dense(1)(xx)
Fx = k.models.Model(x, y)
user_24554940
Related