How to create a new column of datatype map<string,string> in pyspark

user_13465436 0
Viewed 46

I want to create a new column(not from an existing columns) having MapType(StringType(),StringType()) how do I achieve this.

So far, I was able to write the below code but it's not working.

df = df.withColumn("normalizedvariation",lit(None).cast(MapType(StringType(),StringType())))

Also I would like to know the different methods to achieve the same, Thank you.

pyspark apache-spark-sql apache-spark dataframe python
1 Answers

This is one way you can try, newMapColumn here woukld be the name of the Map column. You can see the output I got below. If that is not what you are looking for, please let me know. Thanks!

Also you will have to import the functions using the below line:
from pyspark.sql.functions import col,lit,create_map

enter image description here

user_155452210
Related