I'm using tensorflow 2.1 and tf.keras, and whether adding a custom Metric as a simple function or Metric subclassed instance, during training the custom metric appears only be called at the start of training and not after each epoch as I was expecting. The start of a run is below for example (much larger validation segment than usual as a test), where the metric is printing during update_state() and result() but only until epoch 1. In this case the metric is returning a dummy number that increments on each update call, reaching 2 and no further. Supplied metrics such as BinaryAccuracy do produce varying numbers after each epoch so I assume I'm missing or misunderstanding something. What could be the explanation for the observed behaviour?
update
result
Train on 4418 samples, validate on 4418 samples
Epoch 1/120
update
result
result
3800/4418 [========================>.....] - ETA: 0s - loss: 0.1597 - binary_true_positives: 2.0000 result
4418/4418 [==============================] - 3s 622us/sample - loss: 0.1500 - binary_true_positives: 2.0000 - val_loss: 0.0986 - val_binary_true_positives: 2.0000
Epoch 2/120
4418/4418 [==============================] - 0s 89us/sample - loss: 0.0868 - binary_true_positives: 2.0000 - val_loss: 0.0643 - val_binary_true_positives: 2.0000
SOLVED: After looking further, custom metrics are being produced after each epoch as expected, it's just that the Python code to produce them isn't being executed. It dawned on me that the code was likely transformed to a TensorFlow graph (it is viewable in TensorBoard), and the metrics were generated via executing the compiled graph rather than the original Python code that led to the graph representation being produced.