I try to understand best way how to use autoencoders loss functions.
So the often point is that common loss function consist of KL loss and reconstruction loss.
And what really confuse me is that reconstruction loss is multiplied by some big constant for example
xent_loss = self.original_dim * metrics.binary_crossentropy(self.x, x_decoded_mean)
kl_loss = - 0.5 * K.sum(1 + self.z_log_var - K.square(self.z_mean) - K.exp(self.z_log_var), axis=-1)
vae_loss = K.mean(xent_loss + kl_loss)
from https://github.com/mattiacampana/Autoencoders/blob/master/models/vae.py
or from https://towardsdatascience.com/variational-autoencoders-as-generative-models-with-keras-e0c79415a7eb
Could you explain this? Also what I don't understand is that somehow they use reduce_mean and somehow - reduce_sum aggregate function. Is there some difference in these terms? Woud be it painfull for example to use reduce_sum in KL and reduce_mean in reconstruction function?
Thanks for any response