What is the reason for this behavior of Keras Normalizer layer with 1D features?

user_5599687 0
Viewed 45

When I execute the following Keras code:

from tensorflow import keras
from tensorflow.keras import layers

n_input = 100
n_entries = 10
n_features = 2

features = np.random.random(n_entries*n_input*n_features)
features = features.reshape((n_entries, n_input, n_features))

model = keras.Sequential()
norm = layers.Normalization(-1)
norm.adapt(features)
model.add(norm)
print(norm.output)

I get <KerasTensor: shape=(None, None, 2) dtype=float32 (created by layer 'normalization_28')> with 2D output shape as expected.

However, changing n_features to 1:

from tensorflow import keras
from tensorflow.keras import layers

n_input = 100
n_entries = 10
n_features = 1

features = np.random.random(n_entries*n_input*n_features)
features = features.reshape((n_entries, n_input, n_features))

model = keras.Sequential()
norm = layers.Normalization(-1)
norm.adapt(features)
model.add(norm)
print(norm.output)

I get <KerasTensor: shape=(None, None, None) dtype=float32 (created by layer 'normalization_29')>, with None shape, where I would expect 1.

This breaks the pipeline, making 1D features an exceptional case.

What is the reason for this?

tf.keras keras tensorflow python
1 Answers

I think it has something to do with the way the input shape is inferred and the way the output shape is computed. However, I looked at the source code and could not find anything unusual. As a workaround, you can explicitly define the input_shape so it will not be inferred:

from tensorflow import keras
from tensorflow.keras import layers
import numpy as np

n_input = 100
n_entries = 10
n_features = 1

features = np.random.random(n_entries*n_input*n_features)
features = features.reshape((n_entries, n_input, n_features))

model = keras.Sequential()
norm = layers.Normalization(-1, input_shape=(None, n_features))
norm.adapt(features)
model.add(norm)
print(norm.output)

KerasTensor(type_spec=TensorSpec(shape=(None, None, 1), dtype=tf.float32, name=None), name='normalization_33/truediv:0', description="created by layer 'normalization_33'")
user_96578610
Related