How should I setup script-based source generation with CMake?

user_1593077 0
Viewed 47

I'm managing the build of a certain C++ repository using CMake. In this repository, and among other things, I have a bunch of .hpp files in a directory, and for each of these, I need to compile a generated bit of source code, which depends on its contents.

Naively, I would do this by generating a correspond .cpp file, and include that file in the source files of the CMake target for the library or executable I'm building. Now, since I don't actually need the source files themselves, I could theoretically just arrange for the compiler to get its source from the command-line instead.

My question: How would I set up this source generation and compilation, using CMake as idiomatically as possible?

Notes:

  • Assume I have, say, a bash script which can read the .hpp and generate the .cpp on the standard output.
  • CMake version 3.24 or whichever you like.
  • Should work on Unix-like operating systems, and hopefully on Windows-like OSes other than the fact that the bash script will fail.
  • Please comment if additional information is necessary to answer my question.
buildconfiguration code-generation cmake c++
1 Answers

Let's assume for the sake of portability that you have a Python script, rather than a bash script that manages your code generation. Let's say that it takes two arguments: the source .hpp file and the destination .cpp file. We'll assume it is in your source tree under ./tools/codegen.py.

Now let's assume that your .hpp files are in ./src/genmod for "generated module" because the sources for these headers are generated by codegen.py.

Finally, we'll assume there's a final executable target, app, with a single source file, ./src/main.cpp.

Here's a minimal build that will work for this, with some step-by-step discussion.

We start with some boring boilerplate.

cmake_minimum_required(VERSION 3.24)
project(example)

This probably works on earlier versions, I just haven't tested it, so YMMV. Now we'll create the executable target and link it to our generated sources preemptively. Note that the dependent target does not need to exist before calling target_link_libraries.

add_executable(app src/main.cpp)
target_link_libraries(app PRIVATE genmod)

Now we'll find a Python interpreter and write down an absolute path to the codegen tool.

find_package(Python3 REQUIRED)
set(codegen_py "${CMAKE_CURRENT_SOURCE_DIR}/tools/codegen.py")

Next we'll construct the list of input headers. I'm imagining there are three: A.hpp, B.hpp, and C.hpp.

set(input_headers A.hpp B.hpp C.hpp)
list(TRANSFORM input_headers PREPEND src/genmod/)

I used list(TRANSFORM) here to save some typing. Now we'll just create an object library called genmod, which will "hold" the objects for the generated C++ files.

add_library(genmod OBJECT)

And now comes the real meat. For each of the headers, ...

foreach (header IN LISTS input_headers)

we'll construct absolute paths to the header and generated source files ...

    string(REGEX REPLACE "\\.hpp$" ".cpp" gensrc "${header}")
    set(header "${CMAKE_CURRENT_SOURCE_DIR}/${header}")
    set(gensrc "${CMAKE_CURRENT_BINARY_DIR}/${gensrc}")

and then write a custom command that knows how to call codegen.py. We specify the outputs, command arguments, and dependencies. Don't forget to include the generator script as a dependency, and never forget to pass VERBATIM to ensure consistent, cross-platform, argument quoting.

    add_custom_command(
        OUTPUT "${gensrc}"
        COMMAND Python3::Interpreter "${codegen_py}" "${header}" "${gensrc}"
        DEPENDS "${header}" "${codegen_py}"
        VERBATIM
    )

Finally, we attach this source to genmod.

    target_sources(genmod PRIVATE "${gensrc}")
endforeach ()

We can test this build using Ninja's dry-run feature to make sure the commands look correct.

$ cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python3: /home/reinking/.venv/default/bin/python3.9 (found version "3.9.2") found components: Interpreter 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/reinking/test/build

$ cmake --build build -- -nv
[1/8] cd /home/reinking/test/build && /home/reinking/.venv/default/bin/python3.9 /home/reinking/test/tools/codegen.py /home/reinking/test/src/genmod/A.hpp /home/reinking/test/build/src/genmod/A.cpp
[2/8] cd /home/reinking/test/build && /home/reinking/.venv/default/bin/python3.9 /home/reinking/test/tools/codegen.py /home/reinking/test/src/genmod/B.hpp /home/reinking/test/build/src/genmod/B.cpp
[3/8] cd /home/reinking/test/build && /home/reinking/.venv/default/bin/python3.9 /home/reinking/test/tools/codegen.py /home/reinking/test/src/genmod/C.hpp /home/reinking/test/build/src/genmod/C.cpp
[4/8] /usr/bin/c++   -O3 -DNDEBUG -MD -MT CMakeFiles/genmod.dir/src/genmod/A.cpp.o -MF CMakeFiles/genmod.dir/src/genmod/A.cpp.o.d -o CMakeFiles/genmod.dir/src/genmod/A.cpp.o -c /home/reinking/test/build/src/genmod/A.cpp
[5/8] /usr/bin/c++   -O3 -DNDEBUG -MD -MT CMakeFiles/genmod.dir/src/genmod/B.cpp.o -MF CMakeFiles/genmod.dir/src/genmod/B.cpp.o.d -o CMakeFiles/genmod.dir/src/genmod/B.cpp.o -c /home/reinking/test/build/src/genmod/B.cpp
[6/8] /usr/bin/c++   -O3 -DNDEBUG -MD -MT CMakeFiles/genmod.dir/src/genmod/C.cpp.o -MF CMakeFiles/genmod.dir/src/genmod/C.cpp.o.d -o CMakeFiles/genmod.dir/src/genmod/C.cpp.o -c /home/reinking/test/build/src/genmod/C.cpp
[7/8] /usr/bin/c++   -O3 -DNDEBUG -MD -MT CMakeFiles/app.dir/src/main.cpp.o -MF CMakeFiles/app.dir/src/main.cpp.o.d -o CMakeFiles/app.dir/src/main.cpp.o -c /home/reinking/test/src/main.cpp
[8/8] : && /usr/bin/c++ -O3 -DNDEBUG  CMakeFiles/genmod.dir/src/genmod/A.cpp.o CMakeFiles/genmod.dir/src/genmod/B.cpp.o CMakeFiles/genmod.dir/src/genmod/C.cpp.o CMakeFiles/app.dir/src/main.cpp.o -o app   && :

And indeed we can see that the commands are what we'd naturally expect them to be.

user_21379960
Related