SparkML - Getting several metrics at the same time with RegressionEvaluator()

user_1773841 0
Viewed 1928

I am trying to evaluate a Gradient-Boosted Tree Regression model using RegressionEvaluator(). I would like to compare the four possible metrics for this evaluator:

  • rmse
  • mse
  • r2
  • mae

This is how I am currently approaching the task.

//PREDICTION AND METRICS FOR GBT
val predictions = cvGBTModel.transform(test)

//Root Mean Squared Error
val evaluatorRMSE = new RegressionEvaluator()
  .setLabelCol("label")
  .setPredictionCol("prediction")
  .setMetricName("rmse");
val rmse = evaluatorRMSE.evaluate(predictions);

//Mean Squared Error
val evaluatorMSE = new RegressionEvaluator()
  .setLabelCol("label")
  .setPredictionCol("prediction")
  .setMetricName("mse");
val mse = evaluatorMSE.evaluate(predictions);

//Regression through the origin
val evaluatorR2 = new RegressionEvaluator()
  .setLabelCol("label")
  .setPredictionCol("prediction")
  .setMetricName("r2");
val r2 = evaluatorR2.evaluate(predictions);

//Mean absolute error
val evaluatorMAE = new RegressionEvaluator()
  .setLabelCol("label")
  .setPredictionCol("prediction")
  .setMetricName("mae");
val mae = evaluatorMAE.evaluate(predictions);


print("Root Mean Squared Error (RMSE) on test data = " + rmse);
print("Mean squared error (MSE) on test data = " + mse);
print("Regression through the origin(R2) on test data = " + r2);
print("Mean absolute error (MAE) on test data = " + mae);

Is it possible to get the four metrics at the same time without running 4 different evaluators?

As an aside, I found a similar question where the user found out that RegressionEvaluator is implemented with a RegressionMetrics, which should already contain the four metrics I am looking for. But it is not clear for me how to access these metrics from the evaluator.

apache-spark-mllib apache-spark scala
1 Answers

As zero323 has mentioned, there is no trick that could be used to extract more than one metric

But you can simplify your code. Here are two ways to do that:

val evaluator = new RegressionEvaluator()
    .setLabelCol("label")
    .setPredictionCol("prediction")

val rmse = evaluator.setMetricName("rmse").evaluate(predictions)
println(s"rmse = $rmse")

val mae = evaluator.setMetricName("mae").evaluate(predictions)
println(s"mae = $mae")

val r2 = evaluator.setMetricName("r2").evaluate(predictions)
println(s"r2 = $r2")

Or

import org.apache.spark.ml.param.ParamMap

val evaluator = new RegressionEvaluator()
        .setLabelCol("label")
        .setPredictionCol("prediction")

val rmse = evaluator.evaluate(predictions, ParamMap(evaluator.metricName -> "rmse"))
println(s"rmse = $rmse")

val mae = evaluator.evaluate(predictions, ParamMap(evaluator.metricName -> "mae"))
println(s"mae = $mae")

val r2 = evaluator.evaluate(predictions, ParamMap(evaluator.metricName -> "r2"))
println(s"r2 = $r2")

Resources: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/evaluation/RegressionEvaluator.html (see evaluate method) https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.evaluation.RegressionEvaluator.html (see Examples section)

Admin1
Related