How to create a Keras layer to do a 4D convolutions (Conv4D)?

user_2406793 0
Viewed 1674

It seems that tf.nn.convolution should be able to do 4D convolutions, but I haven't been able to succeed in creating a Keras layer to use this function.

I have tried using Keras Lambda layer to wrap the tf.nn.convolution function but maybe somebody else has a better idea?

I want to exploit the higher-dimensional structure of my data, so reshaping might not be able to capture the nature of my dataset.

conv-neural-network keras tensorflow convolution
2 Answers

Super cool question.

This needs a custom layer (with trainable parameters).
The following accepts any number of dimensions, you control that via kernel_size. 

class Conv(Layer):
    def __init__(self, filters, kernel_size, padding='VALID', **kwargs):
        self.filters = filters
        self.kernel_size = kernel_size #must be a tuple!!!!
        self.padding=padding

        super(Conv, self).__init__(**kwargs)

    #using channels last!!!
    def build(self, input_shape):
        spatialDims = len(self.kernel_size)
        allDims = len(input_shape)
        assert allDims == spatialDims + 2 #spatial dimensions + batch size + channels

        kernelShape = self.kernel_size + (input_shape[-1], self.filters)
            #(spatial1, spatial2,...., spatialN, input_channels, output_channels)

        biasShape = tuple(1 for _ in range(allDims-1)) + (self.filters,)


        self.kernel = self.add_weight(name='kernel', 
                                      shape=kernelShape
                                      initializer='uniform',
                                      trainable=True)
        self.bias = self.add_weight(name='bias', 
                                    shape = biasShape, 
                                    initializer='zeros',
                                    trainable=True)
        self.built = True

    def call(self, inputs):
        results = tf.nn.convolution(inputs, self.kernel, padding=self.padding)
        return results + self.bias

    def compute_output_shape(self, input_shape)
        sizes = input_shape[1:-1]

        if self.padding='VALID' or self.padding='valid':
            sizes = [s - kSize + 1 for s, kSize in zip(sizes, self.kernel_size)]

        return input_shape[:1] + sizes + (self.filters,)
user_20972400

You are right there is not much support for Tensorflow Conv4D right now. I coded this repo (https://github.com/Vincentx15/Conv4D) that addresses that and includes flexible striding and padding.

It adds a lightweight dependency on Sonnet to use advanced padding, but this should blend in tensorflow code seamlessly.

Best :)

user_130709700
Related