I am loading (no action triggered) a csv file into a dataframe in databricks community edition. When giving inferSchema option as true, spark is triggering 2 jobs as expected. But the second job is having 8 tasks. Which configuration decides how many tasks spark should trigger to infer the schema? I tried to change the spark.default.parallelism to different values, but it seems to have no impact on the tasks to infer schema.
