Monitoring GPU usage

If you (like me) happen to be the performance freak, most likely you are well aware of process viewers like htop. Since I’ve started working with GPU-computing I missed htop-like tool tailored to monitor GPU usage. This is becoming more of an issue if you’re working in multi-GPU setups.

You can use `nvidia-smi` which is shipped with NVIDIA drivers, but it’s not very interactive.

gpustat provide nice and interactive view of the processes running and resources used across your GPUs, but you’ll need to switch between windows if you want to also monitor CPU usage.

pip install -U gpustat
gpustat -i

Some time ago I’ve discovered glances – really powerful htop replacement. What’s best about glances (at least for me) is that beside I/O and information from sensors, you can see GPU usage. This is done thanks to py3nvml.

pip install -U glances py3nvml
glances

At first glances window may look a bit overwhelming, but after a few uses you’ll likely fell in love with it!

And what’s your favorite GPU process viewer?

Multiprocessing in Python and garbage collection

Working with multiple threads in Python¬†often leads to¬†high RAM consumption. Unfortunately, automatic garbage collection in child processes isn’t working well. But there are two alternatives:

  • When using Pool(), you can specify no. of task after which the child will be restarted resulting in memory release.
p = Pool(processes=4, maxtasksperchild=1000)
  • If you use Process(), you can simply delete unwanted objects call gc.collect() inside the child. Note, this may slow down your child process substantially!