Add other metrics to compute performance

Question

I use TFF version 0.12.0 In order to compute performance of model, I would like to add (with accuracy ) sensitivity and specificity metrics,

def specificity
...
def create_compiled_keras_model():
    ....

    model.compile(optimizer=tf.keras.optimizers.SGD(lr=0.001, momentum =0.9), 
             loss=tf.keras.losses.BinaryCrossentropy(),
              metrics=([tf.keras.metrics.BinaryAccuracy()], sensitivity, specificity))
    return model

I found this error:

TypeError: Type of `metrics` argument not understood. Expected a list or dictionary, found: ([], , )

So how can I add metrics in Tensorflow federated Thanks

Zachary_Garrett_14692 · Answer

TFF requires metrics to be implemented using the tf.keras.metrics.Metric interface, and can't wrap arbitrary Python functions.

An example of making a custom metric based off of the tf.keras.metrics.Sum subclass can be found in https://github.com/tensorflow/federated/blob/3ed93c8036501fe327ede249a4b0f20d02c6f476/tensorflow_federated/python/learning/keras_utils_test.py#L33. The key being the implementation of update_state method.

For sensitivity and specificity metrics, looking at the implementation of tf.keras.metrics.SensitivityAtSpecificity and it's base class tf.keras.metrics.SensitivitySpecificityBase might be useful examples.

Add other metrics to compute performance

1 Answers