tf.Keras custom layer output shape is None

user_8167107 0
Viewed 2641

I was building a custom layer and encounter output shape problem when adding a dense layer afterward. The output shape of that layer seems doesn't seem to be defined, even if I explicitly do so. Here is a minimal code that is reproducing the issue:

import tensorflow as tf
from tensorflow import keras

class fakeLayer(keras.layers.Layer):

    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    def compute_output_shape(self, input_shape):
        return ((input_shape[0], input_shape[1]* input_shape[2], input_shape[3]))

    def build( self, input_shape):
        super().build(input_shape)

    def call(self, inputs):
        return(tf.reshape(inputs , self.compute_output_shape(tf.shape(inputs))))


inp = keras.layers.Input((32,32,3))
x = keras.layers.Conv2D(16, (3,3))(inp)
x = fakeLayer()(x)
# x = keras.layers.Flatten()(x)
# x = keras.layers.Dense(1)(x)
model = keras.models.Model(inputs= inp, outputs = x)

print(model.summary())

ouputs this :

WARNING:tensorflow:Entity <bound method fakeLayer.call of <__main__.fakeLayer object at 0x0000021A7370E470>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: 
WARNING: Entity <bound method fakeLayer.call of <__main__.fakeLayer object at 0x0000021A7370E470>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: 
Model: "model_9"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_37 (InputLayer)        [(None, 32, 32, 3)]       0         
_________________________________________________________________
conv2d_10 (Conv2D)           (None, 30, 30, 16)        448       
_________________________________________________________________
fake_layer_28 (fakeLayer)    (None, None, None)        0         
=================================================================
Total params: 448
Trainable params: 448
Non-trainable params: 0
_________________________________________________________________
None

We can see here that model.summary() can't get the output shape of the layer.

We thus obtain this error while uncommenting the flatten and dense layer:

WARNING:tensorflow:Entity <bound method fakeLayer.call of <__main__.fakeLayer object at 0x0000021A737535C0>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: 
WARNING: Entity <bound method fakeLayer.call of <__main__.fakeLayer object at 0x0000021A737535C0>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: 
Traceback (most recent call last):

  File "C:\Users\integration\Documents\Scripts\minimalBug.py", line 31, in <module>
    x = keras.layers.Dense(1)(x)

  File "C:\Users\integration\.conda\envs\py36\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 817, in __call__
    self._maybe_build(inputs)

  File "C:\Users\integration\.conda\envs\py36\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 2141, in _maybe_build
    self.build(input_shapes)

  File "C:\Users\integration\.conda\envs\py36\lib\site-packages\tensorflow_core\python\keras\layers\core.py", line 1015, in build
    raise ValueError('The last dimension of the inputs to `Dense` '

ValueError: The last dimension of the inputs to `Dense` should be defined. Found `None`.
keras tensorflow
2 Answers

I don't know if it makes a difference, but you have double parentheses here, don't know why:

def compute_output_shape(self, input_shape):
    #wrong: return ((input_shape[0], input_shape[1]* input_shape[2], input_shape[3]))
    return (input_shape[0], input_shape[1]* input_shape[2], input_shape[3])

Now, the real problem is in call, where you're mixing a lot of different things.

  • tf.shape returns a tensor, not a tuple
  • compute_output_shape returns a tuple
  • if you give a tensor to compute_output_shape it will return a tuple of tensors (never do this!)
  • tf.reshape accepts either "a tuple" or "a tensor", it does not accept a tuple of tensors
  • Keras shapes are tuples that contain None values, reshape accepts tuples with -1 values, not None.

Solution:

def call(self, inputs):
    keras_shape = keras.backend.int_shape(inputs) #tuple containing None and numbers
    tf_shape_tuple = tuple(-1 if s is None else s for s in keras_shape) #None -> -1

    return tf.reshape(inputs , tf_shape_tuple)
user_20972400

I would use tf.shape(x) which deals Ok with the problem of the None shape (source):

def call(self, inputs):
    in_shape = tf.shape(inputs)
    return tf.reshape(inputs , (in_shape[0], in_shape[1]*in_shape[2], in_shape[3]))

For the compute output shape I would first redefine the input shape as:

from tensorflow.python.framework import tensor_shape

def compute_output_shape(self, input_shape):
    input_shape = tensor_shape.TensorShape(input_shape).as_list()
    return (input_shape[0], input_shape[1]* input_shape[2], input_shape[3])
user_59316720
Related