How to test a program that uses Boost.MPI with Boost.Test?

user_465139 0
Viewed 166

I would like to unit-test with Boost.Test a parallel program that uses Boost.MPI. The official documentation of these libraries don't provide instructions on how to do this (or I could not find them). The problem is that MPI code needs initialisation and finalisation done only once which Boost.MPI encapsulates in the environment (and to some extent in the communicator) objects, RAII style. So a Boost.MPI program would normally look like this:

#include <boost/mpi/environment.hpp>
#include <boost/mpi/communicator.hpp>

int main(int argc, char *argv[]) {
  boost::mpi::environment env(argc, argv);
  boost::mpi::communicator world{};
  
  // ... parallel code here ...
}

However, Boost.Test supplies its own main() function behind the scenes. This I can override by defining #define BOOST_TEST_NO_MAIN and then writing the main() myself, along the lines sketched above.

But then the problems begin. How to tell Boost.Test to run the test cases under MPI? Should they be defined as BOOST_AUTO_TEST_CASE(...) macros as usual or is there another way? And how can one "pass" the communicator object (world) to the test cases? Should it be set up in a global fixture? If yes, then how do global fixtures work together with a user-supplied main?

I have experimented a lot, Googled even more, and found nothing usable. Is there anyone out there who knows how to use these two Boost libraries together? If yes, then a working example would be much appreciated. Thanks.

EDIT SebastianH-s suggestion quickly slapped together:

#define BOOST_TEST_MODULE mpitest
#define BOOST_TEST_NO_MAIN

#include <boost/mpi/environment.hpp>
#include <boost/mpi/communicator.hpp>
#include <iostream>

namespace bmpi = boost::mpi;

// Fixture
struct Fix {
    Fix(): env{}, world{} {}
    bmpi::environment env;
    bmpi::communicator world;
};

BOOST_FIXTURE_TEST_CASE(check_mpi, Fix) {
    if (world.rank() == 0) {
        std::cout << "MPI thread level: " <<
        bmpi::environment::thread_level() << std::endl;
    }
    std::cout << "I am process " << world.rank() << " of " << world.size()
            << "." << std::endl;
}

int main(int argc, char *argv[]) {
    // This runs the test cases.
    int retval = boost::unit_test::unit_test_main([](){ return true; }, argc, argv );
}

Compiles with g++ 9.3 and Boost 1.71 under Ubuntu. But when I run it:

Running 1 test case...
*** The MPI_Comm_set_errhandler() function was called before MPI_INIT was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.

See, that's the problem: MPI is not correctly initialised.

mpi unit-testing boost c++
1 Answers

I have found two solutions.

Solution 1: Use global configuration

This is quite simple and elegant. The key is to initialise MPI in a Boost.Test global configuration class (which is not the same as a global fixture).

Code:

#include "boost/mpi/environment.hpp"
#include "boost/mpi/communicator.hpp"
namespace bmpi = boost::mpi;

#define BOOST_TEST_MODULE mpitest
#include "boost/test/unit_test.hpp"
namespace bt = boost::unit_test;

#include <iostream>

struct GlobalConfig {
    GlobalConfig():
    env(
        bt::framework::master_test_suite().argc, 
        bt::framework::master_test_suite().argv,
        bmpi::threading::multiple)
    { }
    bmpi::environment env;
};

BOOST_TEST_GLOBAL_CONFIGURATION(GlobalConfig);
BOOST_AUTO_TEST_SUITE(mpitest);

BOOST_AUTO_TEST_CASE(check_mpi) {
    bmpi::communicator world;   // refers to global MPI_COMM_WORLD instance
    BOOST_CHECK(world.size() > 1);
    
    auto myrank = world.rank();
    if (myrank == 0) {
        std::cout << "MPI thread level: " << bmpi::environment::thread_level() << std::endl;
    }
    std::cout << "I am process " << myrank << " of " << world.size() << std::endl;
}

BOOST_AUTO_TEST_SUITE_END();

Solution 2: Build the test suite manually

This is more complicated because you have to provide a main() function, and put together the test suite manually.

Code:

#include "boost/mpi/environment.hpp"
#include "boost/mpi/communicator.hpp"
namespace bmpi = boost::mpi;

#define BOOST_TEST_NO_MAIN
#define BOOST_TEST_MODULE mpitest
#define BOOST_TEST_DYN_LINK
#include "boost/test/unit_test.hpp"
namespace bt = boost::unit_test;

#include <iostream>

// This is a test case which we will add manually to the test suite.
// see `register_tests()`
void check_mpi() {
    bmpi::communicator world;   // refers to global MPI_COMM_WORLD instance
    BOOST_CHECK(world.size() > 1);
    
    auto myrank = world.rank();
    if (myrank == 0) {
        std::cout << "MPI thread level: " << bmpi::environment::thread_level() << std::endl;
    }
    std::cout << "I am process " << myrank << " of " << world.size() << std::endl;
}

// Put together a test suite manually.
void register_tests() {
    bt::test_suite* ts = BOOST_TEST_SUITE("mpitest");

    // Add a test case
    ts->add( BOOST_TEST_CASE( &check_mpi ));
    // ... add more test cases ...

    // Add the test suite to the "master" test suite
    // which is always provided by the Boost.Test framework
    bt::framework::master_test_suite().add(ts);
}

// == MAIN ==

int main(int argc, char *argv[]) {
    bmpi::environment env(argc, argv, bmpi::threading::multiple);
    bmpi::communicator world;

    register_tests();

    // Run the master test suite.
    // The lambda first argument has to be used if BOOST_TEST_DYN_LINK is set
    int retval = bt::unit_test_main([](){ return true; }, argc, argv );
    return retval;
}

Save either program as mpitest.cc, compile (under Ubuntu 20.04 LTS with g++ 9.3 and Boost 1.71) as follows:

mpicxx -std=c++17  mpitest.cc -DBOOST_TEST_DYN_LINK \
   -L/usr/lib/x86_64-linux-gnu -lboost_unit_test_framework \
   -lboost_mpi -lboost_serialization  -o mpitest

Run as follows:

mpirun -np 2 ./mpitest

Expected output:

Running 1 test case...
Running 1 test case...
I am process 1 of 2
MPI thread level: multiple
I am process 0 of 2

*** No errors detected
*** No errors detected
user_4651390
Related