Run tensorflow model before training, otherweise it won't train?

user_6257330 0
Viewed 61

I found a very interesting thing today when running a tensorflow code:

import matplotlib
from matplotlib import pyplot as plt
import tensorflow as tf
matplotlib.rcParams['figure.figsize'] = [9, 6]

x = tf.linspace(-2., 2., 201)
def f(x):
  y = x**2 + 2*x - 5
  return y
y = f(x) + tf.random.normal(shape=[201])

class Model(tf.keras.Model):
  def __init__(self, units):
    super().__init__()
    self.dense1 = tf.keras.layers.Dense(units=units,
                                        activation=tf.nn.relu,
                                        kernel_initializer=tf.random.normal,
                                        bias_initializer=tf.random.normal)
    self.dense2 = tf.keras.layers.Dense(1)

  def call(self, x, training=True):
    # For Keras layers/models, implement `call` instead of `__call__`.
    x = x[:, tf.newaxis]
    x = self.dense1(x)
    x = self.dense2(x)
    return tf.squeeze(x, axis=1)

model = Model(64)

test = model(x) ################## model couldn't train without this line ####

variables = model.variables
optimizer = tf.optimizers.SGD(learning_rate=0.001)

for step in range(1000):
  with tf.GradientTape() as tape:
    prediction = model(x)
    error = (y-prediction)**2
    mean_error = tf.reduce_mean(error)
  gradient = tape.gradient(mean_error, variables)
  optimizer.apply_gradients(zip(gradient, variables))

  if step % 100 == 0:
    print(f'Mean squared error: {mean_error.numpy():0.3f}')

The model itself is very simple. The interesting thing is the commented line. Without calling the model once, say through test = model(x), the model won't train at all!!! For example, if I remove this line. the results will be:

Mean squared error: 21.782
Mean squared error: 21.782
Mean squared error: 21.782
Mean squared error: 21.782
Mean squared error: 21.782
Mean squared error: 21.782
Mean squared error: 21.782
Mean squared error: 21.782
Mean squared error: 21.782
Mean squared error: 21.782

Why is this line necessary?

keras tensorflow machine-learning python
1 Answers

Before you can train your model, you must build it and compile it.

Building the model creates all variables of the model depending on the input_shape of your training data.

Compiling the model sets the optimizer and loss function you would like to use during training.

When you call the model, it is automatically built with the shape of the data you inserted. Because of that, you can train the model, after you have called it.

user_179057640
Related