how could I filter rows where any value in a pyspark column which is in array format is greater than 0?

Viewed 18

For example,

I have a pyspark data frame like this:

value             
["9.1","0","0","-1.5","-3","9.456"]   
["9.456","4","5.345","6","7.34","8"]
["7","4","5","6","7"]
["-1.66"]
["0"]
["-5","4","6"]
["9","4.567"]

The output should be:

value
    ["9.456","4","5.345","6","7.34","8"]
    ["7","4","5","6","7"]
    ["0"]
    ["9","4.567"]

The output should remove the rows where any of the array elements has a value less than 0.

I tried this: but, it didn't work:

is_greater = lambda x: x>0 == 0
df_new = df_previous.withColumn("value", filter(f.col("value"), is_greater))
0 Answers
Related