Python: How to write a script to compile and run a C++ program via CMake?

If the C++ code is not dynamically generated or something, I would recommend you build it upfront, before starting your python program and merely execute it from the python program.

If you have a good reason to build it from the python code, I created you a little example at https://github.com/kyotov/experiments. Something like this works for run_experiements.py:

import os
import subprocess


def main():
    os.makedirs('build', exist_ok=True)
    subprocess.check_call('cmake -B build -G "NMake Makefiles"', shell=True)
    subprocess.check_call('cmake --build .', shell=True, cwd='build')

    subprocess.check_call('experiments 1 2 3', shell=True, cwd='build')
    subprocess.check_call('experiments 2 3 4', shell=True, cwd='build')


if __name__ == '__main__':
    main()

I used the following for main.cpp:

#include <iostream>

int main(int argc, char **argv) {
    for (auto i = 0; i < argc; i++)
        std::cout << argv[i] << std::endl;
    return 0;
}

And when I run python run_experiements.py I get this output:

(base) C:\kamen\clion\experiments>python run_experiments.py
-- Configuring done
-- Generating done
-- Build files have been written to: C:/kamen/clion/experiments/build
[100%] Built target experiments
experiments
1
2
3
experiments
2
3
4

To get this to work, you will need to setup your environment correctly to find the tools you need. In my case I used Miniconda3 for python and then "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvars64.bat" to get my compiler and cmake in the path.

I did not have a Linux machine handy, but if you make the -G conditional and use Unix Makefiles it will likely work on Linux (and elsewhere) too. Note that you don't have to bother with the .exe on Windows, because the shell is smart enough to start the program even if you don't specify it.

Python's subprocess module will enable you to run external programs like CMake and the binary that you build with it. The executable name, however, will be different on Windows and Linux (with Windows having a .exe extension). You should be able to specify ./test_tokenizing to run your program and the system should find it providing you specify the current working directory cwd.

Try this:

import os
import platform
import subprocess
import sys


def main(*args):
    current_directory = os.getcwd()
    build_dir = os.path.join(current_directory, 'build')

    if not os.path.exists(build_dir):
        os.makedirs(build_dir)

    if platform.system() == 'Windows':
        exe = os.path.join(build_dir, 'test_tokenizing.exe')
    else:
        exe = os.path.join(build_dir, 'test_tokenizing')

    cmake_args = ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=' + build_dir]
    cmake_args += ['-G', 'Unix Makefiles']

    subprocess.run(['cmake', ".."] + cmake_args, cwd=build_dir, check=True)
    subprocess.run(['cmake', '--build', '.'], cwd=build_dir, check=True)
    p = subprocess.run([exe, *args], cwd=build_dir, capture_output=True, text=True)
    print(p.stdout)


if __name__ == '__main__':
    args = []
    if len(sys.argv) > 1:
        args += sys.argv[1:]
    main(*args)

(Requires Python 3.7+ for capture_output and text in subprocess.run).

Here you can pass command line arguments to the script like this:

python run_experiments.py hello world

and the arguments are forwarded to the function main(), which will in turn call your C++ executable with them.

Edit:

Just in case ./test_tokenizing doesn't work on some exotic OS or you want to include some platform specific compiler/build flags, I've put a check for the platform above and specified the full path to your C++ executable.