Upsampling by decimal factor in Keras

Question

I want to use an upsampling 2D layer in keras so that I can increase the image size by a decimal factor (in this case from [213,213] to [640,640]). The layer is compiled as expected, but when I want to train or predict on real images, they are upsampled only by the closest integer to the input factor. Any idea? Details below:

Network:

mp_size = (3,3)
inputs = Input(input_data.shape[1:]) 
lay1 = Conv2D(32, (3,3), strides=(1,1), activation='relu', padding='same', kernel_initializer='glorot_normal')(inputs)
lay2 = MaxPooling2D(pool_size=mp_size)(lay1)
lay3 = Conv2D(32, (3,3), strides=(1,1), activation='relu', padding='same', kernel_initializer='glorot_normal')(lay2)
size1=lay3.get_shape()[1:3]
size2=lay1.get_shape()[1:3]
us_size = size2[0].value/size1[0].value, size2[1].value/size1[1].value
lay4 = Concatenate(axis=-1)([UpSampling2D(size=us_size)(lay3),lay1])
lay5 = Conv2D(1, (1, 1), strides=(1,1), activation='sigmoid')(lay4)
model = Model(inputs=inputs, outputs=lay5)

Network summary when I use model.summary() :

____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
input_4 (InputLayer)             (None, 640, 640, 2)   0                                            
____________________________________________________________________________________________________
conv2d_58 (Conv2D)               (None, 640, 640, 32)  608         input_4[0][0]                    
____________________________________________________________________________________________________
max_pooling2d_14 (MaxPooling2D)  (None, 213, 213, 32)  0           conv2d_58[0][0]                  
____________________________________________________________________________________________________
conv2d_59 (Conv2D)               (None, 213, 213, 32)  9248        max_pooling2d_14[0][0]           
____________________________________________________________________________________________________
up_sampling2d_14 (UpSampling2D)  (None, 640.0, 640.0,  0           conv2d_59[0][0]                  
____________________________________________________________________________________________________
concatenate_14 (Concatenate)     (None, 640.0, 640.0,  0           up_sampling2d_14[0][0]           
                                                                   conv2d_58[0][0]                  
____________________________________________________________________________________________________
conv2d_60 (Conv2D)               (None, 640.0, 640.0,  65          concatenate_14[0][0]             
====================================================================================================
Total params: 9,921
Trainable params: 9,921
Non-trainable params: 0

Error when training the network:

InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,639,639,32] vs. shape[1] = [1,640,640,32]
     [[Node: concatenate_14/concat = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](up_sampling2d_14/ResizeNearestNeighbor, conv2d_58/Relu, concatenate_14/concat/axis)]]

user_11530462 · Answer

It can be resolved by using the below code:

from keras.layers import UpSampling2D
from keras.utils.generic_utils import transpose_shape
class UpSamplingUnet(UpSampling2D):
    def compute_output_shape(self, input_shape):
        size_all_dims = (1,) + self.size + (1,)
        spatial_axes = list(range(1, 1 + self.rank))
        size_all_dims = transpose_shape(size_all_dims,
                                        self.data_format,
                                        spatial_axes)
        output_shape = list(input_shape)
        for dim in range(len(output_shape)):
            if output_shape[dim] is not None:
                output_shape[dim] *= size_all_dims[dim]
                output_shape[dim]=int(output_shape[dim])
        return tuple(output_shape)

Then alter UpSampling2D(size=us_size) to UpSamplingUnet(size=us_size).

Upsampling by decimal factor in Keras

1 Answers