TensorFlow is not using GPU despite all prerequisites installed

Question

I have installed CUDA 11.0, cuDNN 8.0.5 (for CUDA 11.0) and tensorflow-gpu 2.4.1. under Windows 10 Pro. I am checking the installation using the following code:

import tensorflow as tf
from tensorflow.python.client import device_lib
print(tf.version)
print(tf.test.is_built_with_cuda())
print()
print(tf.config.list_physical_devices())
print()
print(device_lib.list_local_devices())

The output seems to suggest that TensorFlow sees the GPU:

init.py'>

True

[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'),

PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

[name: "/device:CPU:0" device_type: "CPU" memory_limit: 268435456 locality { } incarnation: 2790105922589511936 , name: "/device:GPU:0" device_type: "GPU" memory_limit: 3173331764 locality { bus_id: 1
links { } } incarnation: 12842674850440179486 physical_device_desc: "device: 0, name: NVIDIA Quadro M1200, pci bus id: 0000:01:00.0, compute capability: 5.0" ]

The relevant entries in PATH:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\extras\CUPTI\libx64

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\include

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\libnvvp

Here is the CUDA_PATH (though probably unnecessary, since I copied .dll, .h and .lib files of cuDNN into the corresponding directories of CUDA):

C:\cudnn-11.0-windows-x64-v8.0.5.39\cuda\bin

However, when I train a convolutional NN, CPU load rockets to almost 100% while GPU load remains at 0%. What do I need to do to make TensorFlow use the GPU?

Update: I tried the following code:

import numpy as np
import tensorflow as tf
tf.debugging.set_log_device_placement(True)

for i in range(100):
    a = tf.constant(np.random.randn(1000, 1000))
    b = tf.constant(np.random.randn(1000, 1000))
    c = tf.matmul(a, b)
    print(c[465][266])

The output (shown partially) says clearly that TensorFlow believes it is using the GPU:

Executing op MatMul in device

/job:localhost/replica:0/task:0/device:GPU:0 Executing op StridedSlice in device /job:localhost/replica:0/task:0/device:GPU:0 Executing op

StridedSlice in device /job:localhost/replica:0/task:0/device:GPU:0 tf.Tensor(48.741279799952544, shape=(), dtype=float64)

Still, NZXT Cam shows only CPU usage. It does show GPU utilization when I use Zoom though, so it is not a problem with NZXT Cam. It seems like TensorFlow thinks that it is using GPU while it is not doing so in reality...

TensorFlow is not using GPU despite all prerequisites installed

0 Answers