I am implementing a spark application, of which below is a sample snippet(Not the exact same code):
val rdd1 = sc.textfile(HDFS_PATH)
val rdd2 = rdd1.map(func)
rdd2.persist(StorageLevel.MEMORY_AND_DISK)
println(rdd2.count)
On checking the performance of this code from the Spark Application Master UI, I see an entry for the count action, but not for the persist. The DAG for this count action also has a node for the 'map' transformation (line 2 of the above code).
Is it safe to conclude that the map transformation is executed when count (in the last line) is encountered, and not when persist is encountered?
Also, at what point is rdd2 actually persisted?
I understand that only two types of operations can be called on RDDs - transformations and actions. If the RDD is persisted lazily when the count action is called, would persist be considered a transformation or an action or neither?