How to add an empty map type column to DataFrame?

user_4829317 0
Viewed 5898

I want to add a new map type column to a dataframe, like this:

|-- cMap: map (nullable = true)
|    |-- key: string
|    |-- value: string (valueContainsNull = true)

I tried the code:

df.withColumn("cMap", lit(null).cast(MapType)).printSchema

The error is:

:132: error: overloaded method value cast with alternatives:
(to: String)org.apache.spark.sql.Column
(to: org.apache.spark.sql.types.DataType)org.apache.spark.sql.Column cannot be applied to (org.apache.spark.sql.types.MapType.type)

Is there another way to cast the new column to Map or MapType?

apache-spark-sql apache-spark dataframe scala dictionary
4 Answers
map().cast("map<string,string>")

To me, lit(null).cast(...) was not an option, because map_concat with non-empty map returned null... while the above worked well.

user_27535010
Related