With Dynamic Resource Allocation :
A Spark application removes an executor when it has been idle for more than spark.dynamicAllocation.executorIdleTimeout seconds.
And when I set the executor idle timeout property
as followed spark.dynamicAllocation.executorIdleTimeout= 300 , it throws the following warning :
spark.ExecutorAllocationManager: Removing executor 0 because it has been idle for 300 seconds (new desired total will be 2)
What does mean to be "Idle"? Does it means the worker use no CPU? Does a blocking call to database count as Idle?