Keras: methods to enlarge spartial dimension of the layer output blob

user_1179925 0
Viewed 4897

What methods can be used to enlarge spartial dimension of the layer output blob?

As far as I can see from documentation it can be:

  1. UpSampling2D Does it only upsample with power of 2? Also it Repeats the rows and columns of the data by size[0] and size[1] respectively. and it's not very smart.

  2. Conv2DTranspose Can it have arbitary output size (not power of 2 upsapmple)?

    How bilinear interpolation upscale with arbitary dimensions can be done (it can be Conv2DTranspose with fixed weights?)

    What other options can be used to enlarge spartial dimension of the layer output blob?

keras-layer conv-neural-network keras deep-learning convolution
2 Answers

Expanding on the answer from y300, here is a complete example of wrapping TensorFlow bilinear image resizing in a Keras Lambda layer:

from keras import Sequential
from keras.layers import Lambda
import tensorflow as tf

def UpSampling2DBilinear(size):
    return Lambda(lambda x: tf.image.resize_bilinear(x, size, align_corners=True))

upsampler = Sequential([UpSampling2DBilinear((256, 256))])

upsampled = upsampler.predict(images)

Note that align_corners=True to get similar performance to other bilinear image upsampling algorithms, as described in this post.

To use bicubic resampling, create a new function and replace resize_bilinear with resize_bicubic.

For an implementation even more similar to UpSampling2D, try this:

from keras import backend as K

def UpSampling2DBilinear(stride, **kwargs):
    def layer(x):
        input_shape = K.int_shape(x)
        output_shape = (stride * input_shape[1], stride * input_shape[2])
        return tf.image.resize_bilinear(x, output_shape, align_corners=True)
    return Lambda(layer, **kwargs)

This will allow you to use name='', input_shape='' and other arguments for Lamba, and allows you to pass an integer stride/upsample amount.

user_9401960

You can define your own resize Layer:

from keras import layers, models, utils
from keras.backend import tf as ktf

class Interp(layers.Layer):
    def __init__(self, new_size, **kwargs):
        self.new_size = new_size
        super(Interp, self).__init__(**kwargs)

    def build(self, input_shape):
        super(Interp, self).build(input_shape)

    def call(self, inputs, **kwargs):
        new_height, new_width = self.new_size
        resized = ktf.image.resize_images(inputs, [new_height, new_width],
                                          align_corners=True)
        return resized

    def compute_output_shape(self, input_shape):
        return tuple([None, self.new_size[0], self.new_size[1], input_shape[3]])

    def get_config(self):
        config = super(Interp, self).get_config()
        config['new_size'] = self.new_size
        return config
user_49885030
Related