Why does model.summary() give shape None when input shape is clear and fixed?

user_12214867 0
Viewed 147

The code below that is adapted from tensorflow:

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train = x_train.reshape(len(x_train), -1)

model = tf.keras.models.Sequential([
  tf.keras.layers.Dense(10)
])
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer='adam',
              loss=loss_fn,
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=1, verbose=0)
model.summary()

gives output Shape (32, 10), whereas this code

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(10)
])
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer='adam',
              loss=loss_fn,
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=1, verbose=0)
model.summary()

gives Output Shape (None, 10).

I'm conscious that 32 means batch size, 10 means the output classes. I'd just like to know where does the None come from when input shape is clear and fixed.

tensorflow
1 Answers

The first dimension is the number of samples (batch_size). Since it should be flexible and work with any number of samples or batch sizes, it is represented as None. So, don't worry about it. Your model does not care about the first dimension.

For example in your case input shape is (28,28) and output is (10). The model considers (None,28,28) and (None,10) shapes as input and output. It means that you can feed to the model any number of samples, but each input sample should be (28,28), and the model gives you the same unknown number of samples but each of which with 10 labels. This is the reason that you don't need to set the batch_size in the input_shape parameter in your first layer.

Another example for the first dimension, is when you train your model, vs. when you predict using that model. For training you may pass an input array say (10,28,28), which means 10 samples with 28,28 size. But when you want to get a prediction from your model using model.predict() you may pass one single sample like (1,28,28) to get a prediction. So, The first dimension varies during the model life cycle. So it is set to None.

The first model shows (32,10) because you called it after model.fit() and you didn't specified input_shape in your first layer, so it inferences the shapes from training procedure. model.fit() sets batch_size to 32 as default. So, it shows the batch size.
But if you set input_shape, since you should not include the batch size, model will be created by None as the first dimension.

user_24232780
Related