I already have a tensorflow-gpu 1.14 installed, however I installed it through a pre-built wheel so I have no idea what CUDA/CuDNN version it originally built against. How to find it out?
The reason I ask this question is that in my case I want to install a particular Tensorflow wheel first and install the required CUDA/CuDNN version secondly to support my Tensorflow. If the runtime CUDA/CuDNN library does not match the original CuDA/CuDNN library that the Tensorflow built against, one may have the error "Could not find 'cudart64_80.dll'", or "mismatch of cudnn library, built in 7.1.5, runtime is 7.0.5".
Particular in my case, my error message is "tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda/lib64:/usr/local/cuda-9.0/lib64:"
This error is wierd because libcudnn.so.7 is actually installed and properly linked, see
$ sudo ldconfig -p | grep libcudnn.so.7
libcudnn.so.7 (libc6,AArch64) => /usr/local/cuda/lib64/libcudnn.so.7
libcudnn.so.7 (libc6,AArch64) => /usr/lib/aarch64-linux-gnu/libcudnn.so.7
$ ls -alh /usr/local/cuda/lib64/libcudnn.so.7
lrwxrwxrwx 1 root root 44 Oct 17 19:17 /usr/local/cuda/lib64/libcudnn.so.7 -> /usr/lib/aarch64-linux-gnu/libcudnn.so.7.1.4
$ ls -alh /usr/lib/aarch64-linux-gnu/libcudnn.so.7
lrwxrwxrwx 1 root root 44 Oct 17 17:48 /usr/lib/aarch64-linux-gnu/libcudnn.so.7 -> /usr/lib/aarch64-linux-gnu/libcudnn.so.7.1.4
My debugging processes, (1) I tried to locate the tensorflow package in my system by
$ python -c "import tensorflow as tf; print(tf.__file__)"
***/lib/python3.5/site-packages/tensorflow/__init__.py
(2) I then cd to that directory and tried to find out the hard-coded build inforation
$ cat python/platform/build_info.py
is_cuda_build = True
cuda_version_number = '9.0'
cudnn_version_number = '7.1'
I however if this is True, I still cannot understand why the error happens.
Notes:
1. Refering the original Tensorflow-to-CUDA/CuDNN binding does not work, because my tensorflow is installed through a pre-built wheel.
2. Checking the runtime CUDA/CuDNN does not answer the question, I am asking the library at build phase.
3. Re-installing Tensorflow based on the runtime CUDA/CuDNN is not a preferred solution.