Spark number of Partition returned for Groupby?

Viewed 558

With Spark data frame API how result number of partition is derived for groupby. I tried with simple toy example and did not find its following either parent data frame total number of partitions nor spark.default.parallelism. Image has output with number of partitions for different dataframes: base data frame has total 5 partition and join 200 wheres as group by has 6. iPython Snapshot

0 Answers
Related