Composition of lambda functions in TensorFlow 2.0

user_6842947 0
Viewed 1474

In order to build a big model in TensorFlow 2.0, I am using a functional approach and utilizing the functools module of Python 3.6. I illustrate the problem by showing the code for a specific custom layer.

import tensorflow as tf
import functools
from collections.abc import Iterable


# TODO Check for correctness of the model implementation
class Unit3D(tf.keras.layers.Layer):
    def __init__(self, output_channels,
                 kernel_shape=(1, 1, 1),
                 stride=(1, 1, 1),
                 activation_fn='relu',
                 use_batch_norm=True,
                 use_bias=False,
                 is_training=False,
                 name='unit_3d'):
        super(Unit3D, self).__init__(name=name)
        self._output_channels = output_channels
        self._kernel_shape = kernel_shape
        self._stride = stride
        self._activation = activation_fn
        self._use_batch_norm = use_batch_norm
        self._use_bias = use_bias
        self._is_training = is_training
        self._pipeline = []
        self._pipeline.append(tf.keras.layers.Conv3D(
            filters=self._output_channels,
            kernel_size=self._kernel_shape,
            strides=self._stride,
            padding='same',
            use_bias=self._use_bias,
            data_format='channels_first'
        )
        )
        if self._use_batch_norm:
            bn = tf.keras.layers.BatchNormalization(
                axis=1,
                fused=False,
            )
            bn = functools.partial(bn, training=self._is_training)
            self._pipeline.append(bn)

        if self._activation is not None:
            self._pipeline.append(tf.keras.layers.Activation(
                activation=self._activation
            )
            )

        print(isinstance(self._pipeline, Iterable))
        print(type(self._pipeline))
        self._pipeline = lambda x: functools.reduce(lambda f, g: g(f), self._pipeline, x)

    def call(self, input):
        return self._pipeline(input)

when tested using the following code, it returns the error that

TypeError: reduce() arg 2 must support iteration

The error is related to the composition of functions in self._pipeline in the __init__ method.

import tensorflow as tf
from nets.i3d import Unit3D

model = Unit3D(output_channels=64, kernel_shape=[7,7,7],
               is_training=True)

input = tf.keras.backend.random_uniform(shape=(1,3,64,224,224),
                                        dtype=tf.float32)
output = model(input)

In TensorFlow 2.0, during eager execution all lists are wrapped in a data structure referred to as <class 'tensorflow.python.training.tracking.data_structures.ListWrapper'>

It turns out that this data structure is iterable. I tested it using the Iterable class in collections.abc module.

I am unable to comprehend the issue with this code and am not sure if this is an internal issue in TensorFlow 2.0 , or is it something basic which I am missing over here.

If it helps, I am using tensorflow version 2.0.0-beta1 compiled from source from the r2.0 branch. The corresponding git hash is 8e423e3d56390671f0d954c90f4fd163ab02a9c1.

tensorflow2.0 tensorflow python-3.x
1 Answers

I don't think your problem has anything to do with Tensorflow.

In this line:

self._pipeline = lambda x: functools.reduce(lambda f, g: g(f), self._pipeline, x)

you're overwriting the list that you created earlier in the constructor with a function. Therefore during actual execution of the lambdas (particularly the one that you pass to reduce), it is simply a function.

It becomes more obvious if you refactor the outer lambda it into a regular function for debugging purposes:

    def debug_func(x):
        print(type(self._pipeline))  # prints <class 'function'>
        return functools.reduce(lambda f, g: g(f), self._pipeline, x)

    self._pipeline = debug_func  # Clearly, no longer a list

Solution

I think what you meant to do, is to save the pipeline-as-function in a field different from _pipeline, e.g.:

    # ...
    self._pipeline_func = lambda x: functools.reduce(lambda f, g: g(f), self._pipeline, x)

def call(self, input):
    return self._pipeline_func(input)
user_49407070
Related