How to place checkpoints in a C++ MPI code?

user_12469912 0
Viewed 31

According to Fault-Tolerant Systems (Chapter 6) written by Israel Koren and C. Mani Krishna:

A checkpoint is a snapshot of the entire state of the process at the moment it was taken. It represents all the information that we would need to restart the process from that point. We record the checkpoint on stable storage.

Checkpointing is a common fault-tolerance technique. I intend to test how taking checkpoints work in practice by implementing them in the source code of different MPI programs written in C++. This is my first time implementing checkpoints and I don't know how to do it. I am looking for an example or any reference that can help me. I googled it but didn't find a helpful reference or tutorial. Any suggestion is much appreciated.

c++
1 Answers

C++ and MPI don't provide any built in checkpointing mechanism. the strategy you take may depend on the type of application you are writing. Many scientific codes (that solve PDE/ODE) have explicit time stepping or load stepping that they are doing.

In these cases we tend to analyze what data is needed to restart the algorithm and write that out to disk at the end of each timestep or every n timesteps.

One of the challenges when working with an MPI distributed parallel application in ensuring that you can read the data back in using a different number of ranks or data partition.

One tool that you might find to be interesting for writing the binary files to disk is Adios2.

user_140772120
Related