CUDA incompatible with my gcc version

1. Check the maximum supported GCC version for your CUDA version:

   CUDA version	max supported GCC version
   11.4.1+, 11.5, 11.6, 11.7	11
   11.1, 11.2, 11.3, 11.4.0	10
   11	9
   10.1, 10.2	8
   9.2, 10.0	7
   9.0, 9.1	6
   8	5.3
   7	4.9
   5.5, 6	4.8
   4.2, 5	4.6
   4.1	4.5
   4.0	4.4

2. Set an env var for that GCC version. For example, for CUDA 10.2:

   MAX_GCC_VERSION=8
   

3. Make sure you have that version installed:

   sudo apt install gcc-$MAX_GCC_VERSION g++-$MAX_GCC_VERSION
   

4. Add symlinks within CUDA folders:

   sudo ln -s /usr/bin/gcc-$MAX_GCC_VERSION /usr/local/cuda/bin/gcc 
   sudo ln -s /usr/bin/g++-$MAX_GCC_VERSION /usr/local/cuda/bin/g++
   

   (or substitute /usr/local/cuda with your CUDA installation path, if it's not there)

See this GitHub gist for more information on the CUDA-GCC compatibility table.

If using cmake for me none of the hacks of editing the files and linking worked so I compiled using the flags which specify the gcc/g++ version.
cmake -DCMAKE_C_COMPILER=gcc-6 -DCMAKE_CXX_COMPILER=g++-6 ..

Worked like charm.

If you encounter this error, please read the log file:

$ cat /var/log/cuda-installer.log 
[INFO]: Driver installation detected by command: apt list --installed | grep -e nvidia-driver-[0-9][0-9][0-9] -e nvidia-[0-9][0-9][0-9]
[INFO]: Cleaning up window
[INFO]: Complete
[INFO]: Checking compiler version...
[INFO]: gcc location: /usr/bin/gcc

[INFO]: gcc version: gcc version 9.2.1 20191008 (Ubuntu 9.2.1-9ubuntu2) 

[ERROR]: unsupported compiler version: 9.2.1. Use --override to override this check.

Just follow the suggestion in the log file:

sudo sh cuda_<version>_linux.run --override

Job done :)

I just installed CUDA 10.2 with gcc 9.2 on Kubuntu 19.10 using the --override option.

For CUDA 6.5 (and apparently 7.0 and 7.5), I've created a version of the gcc 4.8.5 RPM package (under Fedora Core 30) that allows that version of gcc to be install alongside your system's current GCC.

You can find all of that information here.

This solved my problem:

sudo rm /usr/local/cuda/bin/gcc
sudo rm /usr/local/cuda/bin/g++
sudo apt install gcc-4.4 g++-4.4
sudo ln -s /usr/bin/gcc-4.4 /usr/local/cuda/bin/gcc
sudo ln -s /usr/bin/g++-4.4 /usr/local/cuda/bin/g++

In my case, I had CUDA already installed from the Ubuntu version and cmake would detect that one instead of the newly installed version using the NVidia SDK Manager.

I ran dpkg -l | grep cuda and could see both versions.

What I had to do is uninstall the old CUDA (version 9.1 in my case) and leave the new version alone (version 10.2). I used the purge command like so:

sudo apt-get purge libcudart9.1 nvidia-cuda-dev nvidia-cuda-doc \
                                nvidia-cuda-gdb nvidia-cuda-toolkit

Please verify that the package names match the version you want to remove from your installation.

I had to rerun cmake from a blank BUILD directory to redirect all the #include and libraries to the SDK version (since the old paths were baked in the existing build environment).

This is happening because your current CUDA version doesn't support your current GCC version. You need to do the following:

1. Find the supported GCC version (in my case 5 for CUDA 9)

   • CUDA 4.1: GCC 4.5
   • CUDA 5.0: GCC 4.6
   • CUDA 6.0: GCC 4.7
   • CUDA 7.0: GCC 4.8
   • CUDA 7.5: GCC 4.8
   • CUDA 8: GCC 5.3
   • CUDA 9: GCC 5.5
   • CUDA 9.2: GCC 7
   • CUDA 10.1: GCC 8

2. Install the supported GCC version

   sudo apt-get install gcc-5
   sudo apt-get install g++-5
   

3. Change the softlinks for GCC in the /usr/bin directory

   cd /usr/bin
   sudo rm gcc
   sudo rm g++
   sudo ln -s /usr/bin/gcc-5 gcc
   sudo ln -s /usr/bin/g++-5 g++
   

4. Change the softlinks for GCC in the /usr/local/cuda-9.0/bin directory

   cd /usr/local/cuda-9.0/bin
   sudo rm gcc
   sudo rm g++
   sudo ln -s /usr/bin/gcc-5 gcc
   sudo ln -s /usr/bin/g++-5 g++
   

5. Add -DCUDA_HOST_COMPILER=/usr/bin/gcc-5 to your setup.py file, used for compilation

   if torch.cuda.is_available() and CUDA_HOME is not None:
       extension = CUDAExtension
       sources += source_cuda
       define_macros += [("WITH_CUDA", None)]
       extra_compile_args["nvcc"] = [
           "-DCUDA_HAS_FP16=1",
           "-D__CUDA_NO_HALF_OPERATORS__",
           "-D__CUDA_NO_HALF_CONVERSIONS__",
           "-D__CUDA_NO_HALF2_OPERATORS__",
           "-DCUDA_HOST_COMPILER=/usr/bin/gcc-5"
       ]
   

6. Remove the old build directory

   rm -rd build/
   

7. Compile again by setting CUDAHOSTCXX=/usr/bin/gcc-5

   CUDAHOSTCXX=/usr/bin/gcc-5 python setup.py build develop
   

Note: If you still get the gcc: error trying to exec 'cc1plus': execvp: no such file or directory error after following these steps, try reinstalling the GCC like this and then compiling again:

sudo apt-get install --reinstall gcc-5
sudo apt-get install --reinstall g++-5

Credits: https://github.com/facebookresearch/maskrcnn-benchmark/issues/25#issuecomment-433382510