If I have only 1 executor with memory 25 GB and if it can run only one task at a time then is it possible to process(transformation and action) 1 TB data if Yes then how it will be read and where intermediate data will be store ?
Also for the same scenario if hadoop file is having 300 input split then there will be 300 partitions in RDD so in this case where will be those partitions ? will it remain on hadoop disk only and my single task will be run 300 times ?