Is there a right_anti when joining in PySpark?

user_11353092 0
Viewed 3026

Looking at the documentation: https://spark.apache.org/docs/latest/api/python/pyspark.sql.html?highlight=join

The "how=" parameter has these options...:

inner, cross, outer, full, fullouter, full_outer, left, leftouter, left_outer, right, rightouter, right_outer, semi, leftsemi, left_semi, anti, leftanti and left_anti.

Join graphs

I know you can flip df1 and df2 and still do a left_anti to achieve a right_anti but is the function parameter just flat out missing a right_anti or am I missing/not understanding a concept?

pyspark apache-spark-sql parameters join function
1 Answers

I think you are not missing the concept. In my opinion it should be available, but the right_anti does currently not exist in Pyspark. Therefore, I would recommend to use the approach you already proposed:

# Right anti join via 'left_anti' and switching the right and left dataframe. 
df = df_right.join(df_left, on=[...], how='left_anti')
user_143387160
Related