Keras model output information/log level

Viewed 4608

I am using Keras to build a neural network model:

model_keras = Sequential()
model_keras.add(Dense(4, input_dim=input_num, activation='relu',kernel_regularizer=regularizers.l2(0.01)))
model_keras.add(Dense(1, activation='linear',kernel_regularizer=regularizers.l2(0.01)))
sgd = optimizers.SGD(lr=0.01, clipnorm=0.5)
model_keras.compile(loss='mean_squared_error',  optimizer=sgd)
model_keras.fit(X_norm_train, y_norm_train, batch_size=20, epochs=100)

The output looks like below. I am wondering if it is possible to out the loss, say every 10 epochs instead of every epoch? Thanks!

Epoch 1/200
20/20 [==============================] - 0s - loss: 0.2661
Epoch 2/200
20/20 [==============================] - 0s - loss: 0.2625
Epoch 3/200
20/20 [==============================] - 0s - loss: 0.2590
Epoch 4/200
20/20 [==============================] - 0s - loss: 0.2556
Epoch 5/200
20/20 [==============================] - 0s - loss: 0.2523
Epoch 6/200
20/20 [==============================] - 0s - loss: 0.2490
Epoch 7/200
20/20 [==============================] - 0s - loss: 0.2458
Epoch 8/200
20/20 [==============================] - 0s - loss: 0.2427
Epoch 9/200
20/20 [==============================] - 0s - loss: 0.2397
Epoch 10/200
20/20 [==============================] - 0s - loss: 0.2367
Epoch 11/200
20/20 [==============================] - 0s - loss: 0.2338
Epoch 12/200
20/20 [==============================] - 0s - loss: 0.2309
Epoch 13/200
20/20 [==============================] - 0s - loss: 0.2281
Epoch 14/200
20/20 [==============================] - 0s - loss: 0.2254
Epoch 15/200
20/20 [==============================] - 0s - loss: 0.2228
   :
2 Answers

It is not possible to reduce frequency of logging to stdout, however, passing verbose=0 argument to fit() method would turn logging completely off.

Since the loop over epochs is not exposed in the Keras' sequential model, one way to collect scalar variable summaries with a custom frequency would be using Keras callbacks. In particular, you could use TensorBoard (assuming you are running with tensorflow backend) or CSVLogger (any backend) callbacks to collect any scalar variable summaries (training loss, in your case):

from keras.callbacks import TensorBoard

model_keras = Sequential()
model_keras.add(Dense(4, input_dim=input_num, activation='relu',kernel_regularizer=regularizers.l2(0.01)))
model_keras.add(Dense(1, activation='linear',kernel_regularizer=regularizers.l2(0.01)))
sgd = optimizers.SGD(lr=0.01, clipnorm=0.5)
model_keras.compile(loss='mean_squared_error',  optimizer=sgd)

TB = TensorBoard(histogram_freq=10, batch_size=20)

model_keras.fit(X_norm_train, y_norm_train, batch_size=20, epochs=100, callbacks=[TB])

Setting histogram_freq=10 will save loss every 10 epochs.

EDIT: passing validation_data=(...) to the fit method will also allow to check validation level metrics.

Create a Keras callback to reduce the number of log lines. By default, Keras print log per every epoch. The following code prints only 10 log lines regardless the number of epochs.

class callback(tf.keras.callbacks.Callback):
  def on_epoch_end(this,Epoch,Logs):
    L = Logs["loss"];

    if Epoch%Lafte==Lafte-1: #Log after a number of epochs
      print(f"Average batch loss: {L:.9f}");
    if Epoch==Epochs-1:
      print(f"Fin-avg batch loss: {L:.9f}"); #Final average

Model = model();
Model.compile(...);

Dsize  = ...   #Number of samples in training data
Bsize  = ...   #Number of samples to process in 1 batch
Steps  = 1000; #Number of batches to use to train
Epochs = round(Steps/(Dsize/Bsize));
Lafte  = round(Epochs/10); #Log 10 times only, regardless of num of Epochs
if Lafte==0: Lafte=1;      #Avoid modulus by zero in on_epoch_end

Model.fit(Data, epochs=Epochs, steps_per_epoch=round(Dsize/Bsize),
          callbacks=[callback()], verbose=0);
Related