Does using Scala implicit classes feature on Spark Dataframe is a monkey patching?

Viewed 143

I'm trying to add side-effect functionality to Spark DataFrame by expanding DataFrame class using Scala implicit classes feature for the reason that "Dataset Transform Method" only allows returning DataFrame.

From Wikipedia - "The term monkey patch ... referred to changing code sneakily – and possibly incompatibly with other such patches – at runtime"

In this post the writer warns from "Monkey Patching with Implicit Classes", but I'm not sure his claims are correct because we are not changing any classes.

Does the following example is potentially "monkey patching" and could somehow be incompatible with Spark future version, or because I'm not overwriting the current DataFrame class and just expanding, it can be no harm?

import org.apache.spark.sql.DataFrame
import org.apache.spark.sql.functions.{col, get_json_object}
    
  object dataFrameSql {
    implicit class DataFrameExSql(dataFrame: DataFrame) {
      def writeDFbyPartition(repartition: Int, output: String): Unit = {
        dataFrame
          .repartition(repartition)
          .write
          .option("partitionOverwriteMode", "dynamic")
          .mode(SaveMode.Overwrite)
          .parquet(output)
      }
    }
  }
0 Answers
Related