How to filter multiple conditions in same column pyspark sql

Viewed 3451

I have a column like below:

+-------+------------------+-------+
|   name|             value|user_id|
+-------+------------------+-------+

| user 1|             view |      1|
| user 1|         processed|      1|
| user 2|             view |      3|
| user 3|             view |      4|
+-------+------------------+-------+

I want to get column like below.

+-------+------------------+-------+
|   name|             value|user_id|
+-------+------------------+-------+
| user 2|             view |      3|
| user 3|             view |      4|
+-------+------------------+-------+

simply, eliminate ids which done both action.

currently, I have two data frame like below

df1 = df.where(value=="processed").select("id").distinct()
df2 = df.where(value=="view").select("id").distinct()

now, how to get only ids which only have view not processed.Or is there any better way to do it?

1 Answers
Related