In spark, can I use full outer join with shuffle hash join strategy? If no, then why?

Viewed 114

I was reading this article which says that we can't use full outer join while using SHJ. I understand this is not possible with BHJ but can't figure out why it should not work with SHJ.

1 Answers

This should answer your question. Technically you can but seems like it was never done but resolved in Spark 3.0.1. But note that by default it would still go for Sort Merge Join for most full outer join. https://issues.apache.org/jira/browse/SPARK-32399

Related