I'm new to Spark, and read a lot of articles about Spark shuffle. Most of them mentioned Spark writes the shuffle files to local disk. What I don't understand is how subsequent remote worker nodes read these shuffle files.
0 Answers
Related
I'm new to Spark, and read a lot of articles about Spark shuffle. Most of them mentioned Spark writes the shuffle files to local disk. What I don't understand is how subsequent remote worker nodes read these shuffle files.