I am trying to decode a string using the inbuilt spark function. It doesn't work. The string is still unreadable.
df.
withColumn("col", decode($"col", "ISO-8859-1"))
I try doing the same using an udf, it works.
val decodeUdf: UserDefinedFunction = udf((name:String) => decode(name))
def decode(name: String): String = {
val s = name.getBytes("ISO-8859-1")
new String(s)
}
df.
withColumn("col", decodeUdf($"col"))
What's the difference?