Keras error on concatenating a list of tensors

user_2891152 0
Viewed 172

I'm reproducing two versions of what I believe to be the same code, but one of them works, the other doesn't:

from tensorflow.keras.layers import Dense, Input, Lambda, concatenate
from tensorflow.keras.models import Model


inp = Input(shape=(9,))

# Version 1 (works)
out_1 = Dense(1)(Lambda(lambda x: x[:,0:4])(inp))
out_2 = Dense(1)(Lambda(lambda x: x[:,4:9])(inp))
out = concatenate([out_1, out_2])
model = Model(inp, out)
model.compile(...)
model.fit(...) ✓


# Version 2 (doesn't work)
out = concatenate([Dense(1)(Lambda(lambda x: x[:,i:j])(inp)) for i, j in [(0, 4), (4, 9)]]) # concatenating with a list comprehension
model = Model(inp, out)
model.compile(...)
model.fit(...) ✗

The error message is:

ValueError: Input 0 of layer dense_2 is incompatible with the layer: expected axis -1 of input shape to have value 4 but received input with shape (None, 5)

I'm not sure if this is a bug or an error on the code, but it looks like the concatenation is mixing the tensors when using a list comprehension. Helps appreciated :)

As a matter of clearing, the compile and fit functions are the same for both cases:

import numpy as np

X = np.random.uniform(0, 1, (100, 9))
Y = np.random.uniform(0, 1, (100, 2))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X, Y)
keras tensorflow python
1 Answers

ValueError: Input 0 of layer dense_2 is incompatible with the layer: expected axis -1 of input shape to have value 4 but received input with shape (None, 5)

I has clearly state for you that the concatenation you are using is on the wrong axis. You has to make it concatenate on the first axis (0) instead of the second one. value 4 means it is looking for the second axis which is [0:4], while you are concatenating [4:9] (which is 5). Try to change the axis parameters, and everything will be fine.

> 
Model: "model_3"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_2 (InputLayer)            [(None, 9)]          0                                            
__________________________________________________________________________________________________
lambda_6 (Lambda)               (None, 4)            0           input_2[0][0]                    
__________________________________________________________________________________________________
lambda_7 (Lambda)               (None, 5)            0           input_2[0][0]                    
__________________________________________________________________________________________________
dense_6 (Dense)                 (None, 1)            5           lambda_6[0][0]                   
__________________________________________________________________________________________________
dense_7 (Dense)                 (None, 1)            6           lambda_7[0][0]                   
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 2)            0           dense_6[0][0]                    
                                                                 dense_7[0][0]                    
==================================================================================================
Total params: 11
Trainable params: 11
Non-trainable params: 0
__________________________________________________________________________________________________
WARNING:tensorflow:From /home/dtlam26/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
Train on 100 samples
100/100 [==============================] - 1s 10ms/sample - loss: 4.1017 - acc: 0.0000e+00

The first version will concatenate on 2 layers, so it does not wrongly concatenate by the axis=-1 by default.

user_137267820
Related