Tensorflow - Extract metric keys and/or dict from tf.contrib.learn.DNNLinearCombinedRegressor

Viewed 317

I am currently having a problem with getting the metrics from the pre-built estimator models.This my code to extract the metrics from the model:

v_metrics = {
    "accuracy":
        tf.contrib.learn.MetricSpec(
            metric_fn=tf.contrib.metrics.streaming_accuracy,
            prediction_key = tf.contrib.learn.PredictionKey.SCORES),
    "precision":
        tf.contrib.learn.MetricSpec(
            metric_fn=tf.contrib.metrics.streaming_precision,
            prediction_key = tf.contrib.learn.PredictionKey.SCORES),
    "recall":
        tf.contrib.learn.MetricSpec(
            metric_fn=tf.contrib.metrics.streaming_recall,
            prediction_key = tf.contrib.learn.PredictionKey.SCORES)
}

v_monitor = tf.contrib.learn.monitors.ValidationMonitor(
                input_fn=lambda: input_fn(df_test),
                eval_steps = 1,
                every_n_steps=10,
                metrics=v_metrics
)


m = build_estimator(model_dir, model_type)
m.fit(input_fn=lambda: input_fn(df_train), steps=train_steps, monitors=[v_monitor])
results = m.evaluate(input_fn=lambda: input_fn(df_test), steps=1)

However, when I run this it gives me:

INFO:tensorflow:Saving dict for global step 28: accuracy = 0.0, global_step = 28, loss = 24.7031, precision = 1.0, recall = 1.0

I looked into the API and the estimator.py and metric_spec.py scripts, but nowhere did I find a way to extract the tensors responsible for the accuracy, precision, and recall. I also looked at other answers, but all them are solutions for custom estimators. I also checked the results on tensorboard and its still 0s and 1s.

Note that when I used the exact same method for tf.contrib.learn.DNNLinearCombinedRegressor with prediction_key = tf.contrib.learn.PredictionKey.CLASSES It works properly.

I am not sure what I am doing wrong with the combined Regressor. I would like to know where to get the proper prediction keys (or what I am doing wrong), so when I run the model, it gives me the proper metric results.

0 Answers
Related