decode vs string.getBytes

Viewed 62

I am trying to decode a string using the inbuilt spark function. It doesn't work. The string is still unreadable.

df.
withColumn("col", decode($"col", "ISO-8859-1"))

I try doing the same using an udf, it works.

val decodeUdf: UserDefinedFunction = udf((name:String) => decode(name))

def decode(name: String): String = {
val s = name.getBytes("ISO-8859-1")
new String(s)
}

df.
withColumn("col", decodeUdf($"col"))

What's the difference?

0 Answers
Related