In the book "Spark - The Definitive Guide" it is written:
When we cache an RDD, we cache the actual, physical data (i.e, the bits). The bits. When this data is accessed again, Spark returns the proper data. This is done through the RDD reference. However, in the Structured API, caching is done based on the physical plan. [...] When we ask for a DataFrame to be cached, Spark will save the data in memory or on disk the first time it computes it.
I understand in both cases the physical data is stored in memory or on disk, but what is the difference between the usage of a RDD reference and a physical plan of a DataFrame? And what advantages does the approach with the physical plan has over the reference?