How to create empty struct in pyspark?

user_4208407 0
Viewed 992

I'm trying to create empty struct column in pyspark. For array this works

import pyspark.sql.functions as F
df = df.withColumn('newCol', F.array([]))

but this gives me an error.

df = df.withColumn('newCol', F.struct())

I saw similar question but for scala not pyspark so it doesn't really help me.

pyspark
1 Answers

If you know the schema of the struct column, you can use the function from_json as follows

    struct_schema = StructType([
       StructField('name', StringType(), False),
       StructField('surname', StringType(), False),
    ])

    df = df.withColumn(
      'newCol', F.from_json(psf.lit(""), struct_schema)
    )
user_51777260
Related