C++: Include directory, but only allow one folder to be visible (for Eigen)

You can create a separate, empty directory include and put a soft link Eigen pointing to myrepo/eigen/Eigen. Then add that include as a search directory, instead of myrepo/eigen.

Git tracks soft links "as is", storing the path of the link in the repository, and recreating the link when cloned. It makes no attempts to verify that the link is valid. As long as:

• the link is given as a relative path to a file/directory within the same repository
• the repository is cloned on a filesystem that is capable of holding soft links

everything will work as intended.

Though the compiler will see only the files that are explicitly given to him, CMake will fetch the CMakeLists.txt from the submodule folder and execute the build steps which can be messy.

CMake is a very powerful build system, so it allows you to handle this problems in an elegant way. CMake has ability to include external project as libraries. This is not same as the simple add_subdirectory(). CMake in the case of external project will actually create another build instance for that project and will use building steps to download, configure, build and install the project before the build can start working on the main target.

Having the external build, you can build the library and install it in a temporary folder (relative to your source, or relative to your build folder). In this way, you can use only the products of the build process from the external project.

Here it's wort mentioning that the external project can use different build system, not necessary CMake, which makes it even more powerful approach.

You can use this small demo I had prepared for Eigen library as external project to add it to your build.

The project has eigen as a submodule in the root of the project initialized by the git submodule add https://gitlab.com/libeigen/eigen.git

CMakeLists.txt

cmake_minimum_required(VERSION 3.17)
project(EigenTest)

include(ExternalProject)

set(CMAKE_CXX_STANDARD 17)
set(EIGEN_INCLUDE eigen_install)

ExternalProject_Add(eigen
        SOURCE_DIR ${CMAKE_SOURCE_DIR}/eigen
        CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${CMAKE_BINARY_DIR} -DINCLUDE_INSTALL_DIR=${EIGEN_INCLUDE}
        )

ExternalProject_Get_Property(eigen install_dir)

add_executable(EigenTest main.cpp)
add_dependencies(EigenTest eigen)
target_include_directories(EigenTest PRIVATE ${install_dir})
target_include_directories(EigenTest PRIVATE ${CMAKE_BINARY_DIR}/${EIGEN_INCLUDE})

main.cpp

#include <iostream>
#include <Eigen/Dense>

using Eigen::MatrixXd;

int main() {
    MatrixXd m(2,2);
    m(0,0) = 3;
    m(1,0) = 2.5;
    m(0,1) = -1;
    m(1,1) = m(1,0) + m(0,1);
    std::cout << m << std::endl;

    return 0;
}

This creates a folder in the build destination that is named eigen_install which will contain the final products of the builded library. Note that in my simple example you should avoid using eigen as the destination, since there will be an eigen folder created by CMake in the build destination during the build of the library. You can control this with additional configuration of the ExternalProject but this is beyond the scope of this question.

Read more about the additional features of ExternalProject module in the CMake documents

Using two powerful concepts like the git submodule and CMake's ExternalProject you can be in a full control of your build process bu selecting which branch of the extrenal project you want to build and with which configuration parameters. At the same time, you will maintain the required separation of the builds (separation of the source codes).

If you cannot do the other answers, for example your filesystem does not support linking, you could copy the "Eigen" directory to a separate directory

file(COPY
  ${CMAKE_CURRENT_SOURCE_DIR}/lib/eigen/Eigen
  ${CMAKE_BUILD_FILES_DIRECTOR}/lib/Eigen
)
// And then link to that folder

include_directories(${CMAKE_BUILD_FILES_DIRECTOR}/lib/Eigen)

I have not tried this code, but in theory it should work. It should be kind of robust, even though its a bit hacky.

cmake documentation

Suppose one wants to include a large open-source project as a submodule

However, for the purpose of using the Eigen library, all one really needs is the contents of the folder

You either want the whole project included as a submodule, or you don't. Submodules are specifically for including whole source trees. If you don't want the whole source tree included verbatim, don't use a submodule.

The usual approach to 3rd-party libraries is to clone their own repo, build their installable artefacts (the public headers and static and/or dynamic libraries), and install them in some 3rd-party library area. Then you compile and link your code against them. The 3rd-party library is an external dependency refernced by your build system, not part of your own repo.

I'd like to share how I solved a similar problem in my last project -- hopefully it will be useful to you. My requirements were similar -- only expose headers and libs the developer of the third party project intended to expose. My project uses cmake itself, but that is not a requirement.

I created a deps folder in the root of my repo and put a CMakeLists.txt in there. This is not to build the main project, but to pull down dependencies I care about, build them (the way their developers intended to be built), and install them for use by my project. Looks like this:

myrepo/
    deps/
        CMakeLists.txt

Similar to @jordanvrtanoski, I use the ExternalProject feature, but I wrote a macro, which allows me to be a bit more succinct. I initially struggled with add_subdirectory and I eventually found his advice to avoid add_subdirectory sound :). Unlike @jordanvrtanoski, I prefer to use a separate CMakeLists.txt for the dependencies, even if my top-level project is cmake based. Otherwise, I have found that cmake will do some checks confirming that the dependencies are properly installed and that robs time from my inner dev loop... I have also found that having the dependencies installed out of the inner loop allows you to reliably do find_package in cmake, and not worry about specifying include and lib directories separately in your top-level project.

Here is a snippet of that CMakeLists.txt

cmake_minimum_required(VERSION 3.17)

project(deps)

include(ExternalProject)

function(install NAME GIT_REPO GIT_TAG)
    ExternalProject_Add(
            ${NAME}
            GIT_REPOSITORY ${GIT_REPO}
            GIT_TAG ${GIT_TAG}
            PREFIX ${CMAKE_SOURCE_DIR}/${CMAKE_BUILD_TYPE}

            ${ARGN}
            CMAKE_ARGS
            --config ${CMAKE_BUILD_TYPE}
            -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
            -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded$<$<CONFIG:Debug>:Debug>
            -DCMAKE_INSTALL_PREFIX:PATH=${CMAKE_SOURCE_DIR}/${CMAKE_BUILD_TYPE}/install
    )
endfunction()

install(gflags
        https://github.com/gflags/gflags.git
        v2.2.0

        CMAKE_ARGS
        -DREGISTER_INSTALL_PREFIX=FALSE
        -DGFLAGS_BUILD_TESTING=FALSE)

install(glog
        https://github.com/google/glog.git
        v0.4.0

        DEPENDS gflags

        CMAKE_ARGS
        -DBUILD_TESTING=FALSE)

You can trigger the installation with:

cmake -DCMAKE_BUILD_TYPE=Debug --log-level=VERBOSE -G "NMake Makefiles" ..
cmake --build . --config Debug

And note that you can do Debug and Release builds separately and the artifacts end up in separate folders under deps.

There is a bit to unpack here but all with a good reason, I hope. You can see that once the macro is out of the way, the snippet to install gflags is pretty minimal. It specifies the repo, the tag, and some gflags-specific options to cmake. The following step to install glog is similarly simple, but I included it as an example of a transitive dependency... you can see that it declares dependency on gflags which is important... Since glog can be installed with or without gflags support and I wanted the latter.

Another, more general thing to note is that the macro overrides the CMAKE_INSTALL_PREFIX for the dependency, so that it gets installed in something like myrepo/deps/Debug/install/.... The installation itself will create include and lib and sometimes bin folder under there depending on how the original developers intended.

When you compile myrepo you need to point the compiler to the install directory, specifically tell it to look for headers in myrepo/deps/Debug/install/include and libraries in myrepo/deps/Debug/install/libs. How you do this depends on the build system you are using for your main project. If it is cmake (like mine) here is what works for me:

...
set(CMAKE_PREFIX_PATH ${CMAKE_SOURCE_DIR}/deps/${CMAKE_BUILD_TYPE}/install)

find_package(gflags REQUIRED NO_MODULE)
find_package(glog REQUIRED NO_MODULE)

add_binary(mybinary
        ...)
target_link_libraries(mybinary      
        glog::glog)

I hope this is useful to you, and I am really looking for feedback from others who have fought this problem. I find the whole dependency infrastructure in C/C++ much more brittle than the newer ones developed for Java (e.g. Maven) or JavaScript (e.g. NPM) or Python (e.g. PyPI), which is honestly somewhat surprising given how long it has been around.

I imagine Eigen in its install script will create Eigen directory under .../deps/Debug/include/Eigen and your dream would come true :)

Note that this approach does not use git submodules, but the sources of the dependencies do end up fetched and included under .../deps/Debug/src by the ExternalProject module... so you can still go there and examine them. You can easily update the revision of the dependency if you so chose by using a different tag or rev in .../deps/CMakeLists.txt and rerunning the install part as described above.

A few parting thoughts and tidbits, which are somewhat important, but did not deserve a place in the main exposition...

1. You can forego my macro in cases where you need to install something special, which is not based on cmake, not stored in git, or even uses prebuilt binaries... The macro was intended to solve the most common happy path for me, but you can use ExternalProject_Add directly in all other cases.

Once I had to once do this for OpenSSL on Windows, and I used this snippet in my deps/CMakeLists.txt:

ExternalProject_Add(
        openssl
        URL https://github.com/CristiFati/Prebuilt-Binaries/raw/master/OpenSSL/v1.1.1/OpenSSL-1.1.1i-Win-pc064.zip
        PREFIX ${CMAKE_SOURCE_DIR}/${CMAKE_BUILD_TYPE}
        CONFIGURE_COMMAND ""
        BUILD_COMMAND ""
        INSTALL_COMMAND ${CMAKE_COMMAND} -E echo installing from `<SOURCE_DIR>/OpenSSL/1.1.1i` to `${CMAKE_SOURCE_DIR}/${CMAKE_BUILD_TYPE}/install`
        COMMAND ${CMAKE_COMMAND} -E copy_directory <SOURCE_DIR>/OpenSSL/1.1.1i/include ${CMAKE_SOURCE_DIR}/${CMAKE_BUILD_TYPE}/install/include
        COMMAND ${CMAKE_COMMAND} -E copy_directory <SOURCE_DIR>/OpenSSL/1.1.1i/lib ${CMAKE_SOURCE_DIR}/${CMAKE_BUILD_TYPE}/install/lib
        COMMAND ${CMAKE_COMMAND} -E copy_directory <SOURCE_DIR>/OpenSSL/1.1.1i/bin ${CMAKE_SOURCE_DIR}/${CMAKE_BUILD_TYPE}/install/bin
)

2. I have had luck with googletest but it prefers static libraries and their cmake scripts do some pretty low-level surgery to inject the right compiler options to do that which confuses things when you have a large set of dependencies, so I had to pass a flag to it to tolerate shared libraries:

install(googletest
        https://github.com/google/googletest.git
        release-1.10.0

        CMAKE_ARGS
        #FIXME: figure out how to make all of them static!!!
        -Dgtest_force_shared_crt=ON)

3. One thing on my todo list is to try to apply this technique to containers and use the standard install directory, not an overriden one like in this example. I will get to it one of these days.

Let me know if you have any questions!