When I use custom training with tf.distribute.Strategy in tensorflow2.4.1 (like that https://www.tensorflow.org/tutorials/distribute/custom_training?hl=en), I want to know is there any fault if the batch size is not divisible.
Specifically, assuming that the dataset has 14 examples, I have 2 GPUs and set the batch size 4, so the training size is ((2,2), (2,2), (2,2), (1,1)).
If I define the loss by
tf.nn.compute_average_loss(per_example_loss, global_batch_size=4),
the loss of last batch will be divided by 4, but it only has 2 examples.