How to concatenate two tensors with different shape in 2d convolution in tensorflow?

user_7302169 0
Viewed 908

In my computational pipeline, I have used custom function which is going to create custom keras blocks, and I used this blocks multiple times with Conv2D. At the end, I got two different tensor which is features maps with different tensor shape: TensorShape([None, 21, 21, 64]) and TensorShape([None, 10, 10, 192]). In this case, using tf.keras.layers.concatenate to do concatenation is not working for me. Can anyone point me out how to concatenate this two tensors into one? Any idea to make this happen?

if I could able to concatenate the tensors with shape of TensorShape([None, 21, 21, 64]) and TensorShape([None, 10, 10, 192]), I want to do the following after the concatenation.

x = Conv2D(32, (2, 2), strides=(1,1), padding='same')(merged_tensors)
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
x = MaxPooling2D(pool_size=(2,2))(x)
x = Flatten()(x)
x = Dense(256)(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Dropout(0.25)(x)
x = Dropout(0.25)(x)
x = Dense(10)(x)
x = Activation('softmax')(x)
outputs = x

model = tf.keras.Model(inputs=inputs, outputs=outputs)

I tried to reshape the tensors with shape of TensorShape([None, 21, 21, 64]) and TensorShape([None, 10, 10, 192]) in 1D convolution and do merge, then reshape the output back to 2d convolution. My way is not working. Can anyone suggest possible way of doing this? Any thoughts?

update

I am still not sure the way of getting output shape of concatenation is going to be TensorShape([None, 21+10, 21+10, 192+64]) or not because I am not sure it does make sense in terms of mathematics standpoint. How to make this concatenation easily and correct? what would be right shape of concatenated one? Any idea?

keras tensorflow python
1 Answers

To operate a concatenation you should provide layers with the same shapes except for the concat axis... in case of images, if you want to concatenate them on features dimensionality (axis -1) the layers must have the same batch_dim, width, and height.

If you want to force the operation you need to do something that equals the dimensionalities. A possibility is the padding. Below an example where I concatenate two layers on the last dimensionality

batch_dim = 32
x1 = np.random.uniform(0,1, (batch_dim, 10,10,192)).astype('float32')
x2 = np.random.uniform(0,1, (batch_dim, 21,21,64)).astype('float32')

merged_tensors = Concatenate()([ZeroPadding2D(((6,5),(6,5)))(x1), x2]) # (batch_dim, 21, 21, 192+64)

with Pooling instead of Padding:

batch_dim = 32
x1 = np.random.uniform(0,1, (batch_dim, 10,10,192)).astype('float32')
x2 = np.random.uniform(0,1, (batch_dim, 21,21,64)).astype('float32')

merged_tensors = Concatenate()([MaxPool2D(2)(x2), x1]) # (batch_dim, 10, 10, 192+64)
user_103750490
Related