I'd like to understand what determines when checkpoints are taken. How does this relate to the checkpointing interval?
I'd like to understand what determines when checkpoints are taken. How does this relate to the checkpointing interval?
To a first approximation, the Checkpoint Coordinator (part of the Job Manager) uses the checkpoint interval to determine when to start a new checkpoint. This interval is passed when you enable checkpointing, e.g., here is it set wait for 10 seconds between checkpoints:
env.enableCheckpointing(10000L);
or it can also be set via execution.checkpointing.interval.
However, the Checkpoint Coordinator will wait however long is necessary to avoid violating either the setting for execution.checkpointing.max-concurrent-checkpoints or for execution.checkpointing.min-pause (which defines how much time must elapse between the completion of one checkpoint and the start of the next one).