What diff `nvidia-smi dmon` between `nvidia-smi` for Memory-Usage metics

user_9953408 0
Viewed 2634

I get nvidia-smi get Memory-Usage is such this

$nvidia-smi -i 0,1
Wed Mar  4 16:20:07 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.113      Driver Version: 418.113      CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce RTX 208...  Off  | 00000000:18:00.0 Off |                  N/A |
| 27%   37C    P8     1W / 250W |  10789MiB / 10989MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce RTX 208...  Off  | 00000000:3B:00.0 Off |                  N/A |
| 41%   50C    P2    67W / 250W |  10893MiB / 10989MiB |      2%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0    231853      C   tensorflow_model_server                    10779MiB |
|    1    120908      C   python                                     10883MiB |
+-----------------------------------------------------------------------------+

This Memory-Usage is 99% ,but When I get nvidia-smi dmon like this

$nvidia-smi dmon -i 0,1
# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0     1    37     -     0     0     0     0   405   300
    1    67    50     -     0     0     0     0  6800  1350
    0     1    37     -     0     0     0     0   405   300
    1    67    50     -     0     0     0     0  6800  1350
    0     1    37     -     0     0     0     0   405   300
    1    67    50     -     0     0     0     0  6800  1350

this mem% has 0%, some time as 0~3% .

Why is there such a difference?

tensorflow nvidia
1 Answers

Memory-Usage from nvidia-smi is the usage of Memory.

mem% from nvidia-smi dmon is the Utilization of Memory.

Memory-Usage = used memory / total memory.

Utilization = time over the past sample period / global (device) memory was being read or written * 100%
user_80132280
Related