Dynamically pass columns into when otherwise functions in Spark SQL

user_727079 0
Viewed 405

Hi I am using withColumn with when otherwise to update a column value

val df2 = df.withColumn("col_1", when(col("col_1") > col("col_2"),col("col_2")).otherwise(col("col_1")))

but my requirement is to make this statement generic so some one can pass a string like "col(\"col_1\") > col(\"col_2\")" and it can be passed to when function this string can have logical operators like &&, ||

Is their a way to pass string expression into Column? Or is their another way to pass these values dynamically?

Thanks

apache-spark-sql apache-spark scala
1 Answers

Perhaps this is helpful - you can pass filter variable directly as below

val filter = "col_1>col_2"
val df2 = df.withColumn("col_1", expr(s"case when ($filter) then col_2 else col_1 end")
user_47588230
Related