how spark select BroadcastExchange side

user_12932522 0
Viewed 109

In my understanding, when Spark selects Broadcast Join, The side with a smaller physical data size will be broadcasted,

but today, use the same sql and table data, I've come across the opposite example, the pic as follow

the smaller table was broadcasted the smaller table was broadcasted

the bigger table was broadcasted the bigger table was broadcasted

so I want to know how spark select the broadcast side, can anyone give me some help ?

apache-spark
1 Answers

If default config is used, spark will not broadcast such a big table, its 2.8 GB is size. Most likely, someone called broadcast on the larger table in code and that's why spark is broadcasting it.

Can you also share the code snippet for the join, can help better then.

user_46595300
Related