Tensorflow 2.0 utilize all CPU cores 100%

user_9280994 0
Viewed 6256

My Tensorflow model makes heavy use of data preprocessing that should be done on the CPU to leave the GPU open for training.

top - 09:57:54 up 16:23,  1 user,  load average: 3,67, 1,57, 0,67
Tasks: 400 total,   1 running, 399 sleeping,   0 stopped,   0 zombie
%Cpu(s): 19,1 us,  2,8 sy,  0,0 ni, 78,1 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
MiB Mem :  32049,7 total,    314,6 free,   5162,9 used,  26572,2 buff/cache
MiB Swap:   6779,0 total,   6556,0 free,    223,0 used.  25716,1 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                
  17604 joro      20   0   22,1g   2,3g 704896 S 331,2   7,2   4:39.33 python

This is what top shows me. I would like to make this python process use at least 90% of available CPU across all cores. How can this be achieved?

GPU utilization is better, around 90%. Even though I don't know why it is not at 100%

Mon Aug 10 10:00:13 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.100      Driver Version: 440.100      CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce RTX 208...  Off  | 00000000:01:00.0  On |                  N/A |
| 35%   41C    P2    90W / 260W |  10515MiB / 11016MiB |     11%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1128      G   /usr/lib/xorg/Xorg                           102MiB |
|    0      1648      G   /usr/lib/xorg/Xorg                           380MiB |
|    0      1848      G   /usr/bin/gnome-shell                         279MiB |
|    0     10633      G   ...uest-channel-token=1206236727             266MiB |
|    0     13794      G   /usr/lib/firefox/firefox                       6MiB |
|    0     17604      C   python                                      9457MiB |
+-----------------------------------------------------------------------------+

All i found was a solution for tensorflow 1.0:

sess = tf.Session(config=tf.ConfigProto(
  intra_op_parallelism_threads=NUM_THREADS))

I have an Intel 9900k and a RTX 2080 Ti and use Ubuntu 20.04

E: When I add the following code on top, it uses 1 core 100%

tf.config.threading.set_intra_op_parallelism_threads(1)
tf.config.threading.set_inter_op_parallelism_threads(1)

But increasing this number to 16 again only utilizes all cores ~30%

tensorflow preprocessor cpu
2 Answers

Just setting the set_intra_op_parallelism_threads and set_inter_op_parallelism_threads wasn't working for me. Incase someone else is in the same place, after a lot of struggle with the same issue, below piece of code worked for me in limiting the CPU usage of tensorflow below 500%:

import os
import tensorflow as tf
num_threads = 5
os.environ["OMP_NUM_THREADS"] = "5"
os.environ["TF_NUM_INTRAOP_THREADS"] = "5"
os.environ["TF_NUM_INTEROP_THREADS"] = "5"

tf.config.threading.set_inter_op_parallelism_threads(
    num_threads
)
tf.config.threading.set_intra_op_parallelism_threads(
    num_threads
)
tf.config.set_soft_device_placement(True)
user_57144890

There can be many issues for this, I solved it for me the following way:

Set tf.config.threading.set_intra_op_parallelism_threads(<Your_Physical_Core_Count>) tf.config.threading.set_inter_op_parallelism_threads(<Your_Physical_Core_Count>)

both to your physical core count. You do not want Hyperthreading for highly vectorized operations as you cannot benefit from parallized operations when there aren't any gaps.

"With a high level of vectorization, the number of execution gaps is very small and there is possibly insufficient opportunity to make up any penalty due to increased contention in HT."

From: Saini et al, published by NASAA dvanced Supercomputing Division, 2011: The Impact of Hyper-Threading on Processor Resource Utilization in Production Applications

EDIT: I am not sure anymore, if one of the two has to be 1. But one 100% needs to be set to Physical.

user_87827780
Related