CUDA driver version is insufficient for CUDA runtime version

Counterintuitively, this error also happens if libcuda.so is not found, even when versions reported by nvidia-smi match perfectly. This library is part of nvidia-drivers package (On CentOS: nvidia-driver-latest-cuda-libs, on Gentoo x11-drivers/nvidia-drivers). It is possible to have the CUDA Tookit with nvcc and libcudart installed and building your app fine, but the drivers part not installed, causing this error.

To diagnose whether this is the reason, use strace:

strace -f -e trace=file ./your_cuda_app

and check for open calls to libcuda.so*, at least one of them should return with a success code, like so:

4928  open("/lib64/libcuda.so.1", O_RDONLY|O_CLOEXEC) = 3 

In my case, I had to run my docker container with nvidia-docker run ... instead of docker run ...

I also had similar problem, updated my graphic driver but the problem still remained. I finally decided to remove Cuda 9.2 and install Cuda 8, it solved my issue.

This is what has worked for me :

1. Installing TensorFlow version 1.14, installing keras
2. Install CUDA 10.0 from https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exenetwork for windows
3. Download the compatible CuDNN i.e 7.4.2 from https://developer.nvidia.com/rdp/cudnn-archive
4. set the following in the path variable cudnn-10.0-windows10-x64-v7.4.2.24\cuda\bin cudnn-10.0-windows10-x64-v7.4.2.24\cuda\include cudnn-10.0-windows10-x64-v7.4.2.24\cuda\lib\x64
5. Restart the Pc

This problem can also be because of incorrect environment setup, e.g. Docker image setup. Although the driver itself is correct, sufficient for your program. If your LD_LIBRARY_PATH points to the wrong driver, it can throw this error. In my case, i get this error when using /usr/local/nvidia/lib/libcuda.so, and if I use /usr/local/nvidia/lib64/libcuda.so everything goes right.