A question about input size in a tensorflow neural network

user_14824108 0
Viewed 426

I'm a PyTorch user but recently reading some code implemented using tensorflow. My question is, if we just have a simple neural network like this, where is the input size specified? Or is this model allowed to work with a variable size input?

class SimpleNet(tf.keras.Model):
    def __init__(self):
        super(SimpleNet, self).__init__()
        activation = tf.keras.activations.tanh
        self.features = [

         #Where is the input size specified?
            tf.keras.layers.Dense(89, activation),
            tf.keras.layers.Dense(6 * 32, activation),
            tf.keras.layers.Dense(32, activation),
            tf.keras.layers.Dense(1),
        ]

    def process(self, x):
        x = apply_layers(x, self.features)
        return x

    ...
tensorflow deep-learning python
1 Answers

The input_shape is inferred when you pass real data to your model. Meaning, the input_shape is variable if you do not explicitly define it. For example, you could explicitly define your input_shape in the first layer of your model:

tf.keras.layers.Dense(89, activation, input_shape=(5,))

And each layer derives the required input shape from the output of the previous layer. Note, however, that once you feed your model with real data, it stores this input_shape for future use. Another example shows that it does not matter what shape the data has as long as it is compatible with the first Dense layer:

import tensorflow as tf

activation = tf.keras.activations.tanh
model1 = tf.keras.Sequential(
    [
    tf.keras.layers.Dense(89, activation),
    tf.keras.layers.Dense(6 * 32, activation),
    tf.keras.layers.Dense(32, activation),
    tf.keras.layers.Dense(1),
    ])

tf.print('Works -->', model1(tf.random.normal((10, 1))).shape)
model2 = tf.keras.Sequential(
    [
    tf.keras.layers.Dense(89, activation),
    tf.keras.layers.Dense(6 * 32, activation),
    tf.keras.layers.Dense(32, activation),
    tf.keras.layers.Dense(1),
    ])
tf.print('Works -->', model2(tf.random.normal((10, 5))).shape)

Works --> TensorShape([10, 1])
Works --> TensorShape([10, 1])

where 10 represents the number of samples and 1 and 5 the number of features.

user_96578610
Related