pyspark.sql 3.1.2 collect_set gives all null values

i am migrating code from pyspark 2.2 to 3.1.2, and i had an issue with collect_set using 3.1.2 pyspark. collect_set gives me all null values instead of unique, pyspark 2.2 works fine with that, idk what the problem, is there a difference between 2.2 and 3.1.2 with collect_set method?

sqlContext.sql(
    select collect_set({column}) over(partition_by({partition_column})) as {col}
)

column contains some regions

or porblem could be in window function?

thanks