How does spark ensure that the spark view is read only once from source in case of 2 parallel actions?

Viewed 15

Following is my code

Dataset data = spark.read().createview("test").cache()
CompletableFuture.allOf(supplyAsync(() -> data.count()), supplyAsync(() -> data.count()))

What happens when two first operations are triggered on the above dataset in parallel? Since the terminal operation will trigger the spark source read. How does spark ensure that the dataset is not read twice from the source since there will be a sort of race conditioning happening?

0 Answers
Related