Is the loss printed by tensorflow a batch/sample wise loss or is it a running average loss?

user_12223614 0
Viewed 135

When I train a TensorFlow model, it usually prints information similar to the below line at each iteration

INFO:tensorflow:loss = 1.9433185, step = 11 (0.300 sec)

Is the loss being printed the loss of the batch that the model saw currently, or is it the running average loss over all the previous batches of the training?

If I use a batch size of 1 i.e. only one training sample in each batch, then the loss printed will be of every sample separately, or will it be a running average loss?

loss-function keras tensorflow machine-learning
1 Answers

The loss reported in the progress bar of Keras/TensorFlow is always a running mean of the batches seen so far, it is not a per-batch value.

I do not think there is a way to see the per-batch values during training.

user_3491300
Related