How is the loss calculated in TensorFlow?

user_11391711 0
Viewed 350

I was reading about the minimum squared error(MSE) in the TensorwFlow (tf) user document.

https://www.tensorflow.org/api_docs/python/tf/keras/metrics/mean_squared_error

When I hard-coded the MSE and print each loss calculated, I observe a different value than what is reported by tf.

import numpy as np
import random
import tensorflow as tf
from tensorflow import keras 

m = 200
n = 5
my_input=  np.random.random([m,n])
my_output =  np.random.random([m,1])


def obj(y_true,y_pred):
    loss = tf.keras.losses.mean_squared_error(y_true,y_pred)
    tf.print("\n")
    tf.print(tf.math.reduce_mean(loss),"\n")
    return (loss)



           

my_model = tf.keras.Sequential([  
    tf.keras.layers.Flatten(input_shape=(my_input.shape[1],)),
    tf.keras.layers.Dense(32, activation='softmax'),
    tf.keras.layers.Dense(32, activation='relu'),
    tf.keras.layers.Dense(1) 
])   
                    

my_model.compile(loss=obj ,optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001))
my_model.fit(my_input,  my_output, epochs=5, batch_size=20, verbose=1)

The loss values match in the first step, however, there is always a difference after that. Could someone explain me the mechanism behind the loss calculation?

Epoch 1/5


0.349255413 

 1/10 [==>...........................] - ETA: 3s - loss: 0.3493

0.449805915 



0.453376621 



0.500476539 



0.294586331 



0.269146353 



0.358534873 

 7/10 [====================>.........] - ETA: 0s - loss: 0.3822
keras tensorflow deep-learning python
2 Answers

By default, the training loss at the end of each epoch is the mean of the batch losses.

In your case, you reported the first seven steps of the first epoch with the corresponding batch losses. So the loss at the seventh step is computed as a simple mean of the losses from the previous steps:

(0.349255413 + 0.449805915 + 0.453376621 + 0.500476539 + 0.294586331 + 0.269146353 + 0.358534873) / 7 = 0.3822
user_103750490

Mean squared error (MSE) is the most commonly used loss function for regression. The loss is the mean overseen data of the squared differences between true and predicted values, or writing it as a formula.

enter image description here

You can use MSE when doing regression, believing that your target, conditioned on the input, is normally distributed, and want large errors to be significantly (quadratically) more penalized than small ones.

According to your example , and as mentioned in the image above :

y_true is y and y_pred is the same as y~i , so it will calculate the loss every epoch in order to get the minimum value that means , y_true will be somehow closer to y_pred

user_118567510
Related