psutil gives other result than resource

user_11935280 0
Viewed 276

I want to check memory consumption of my python code and have therefore added the following rows in the code:

import resource
print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)

As an alternative I have also tried this:

import psutil
process = psutil.Process(os.getpid())
print(process.memory_info().rss)  # in bytes

However, I get different results, as for example 866 480 from resource and 730 689 536 from psutil. Of course as you can see, in the first case it is kilobytes and in the second case bytes, but it is a difference also in addition to that.

Reading the documentation, I still don't understand causes the difference, so input would be valuable.

out-of-memory memory python
1 Answers

TLDR: resource.getrusage sometimes misses that Python already removed objects from memory

There was a bug in memory profiler (which was using resource.getrusage at that time). In this blog post the different methods for memory measurements are described. I cite:

"this approach [resource.getrusage] is several times faster than the one based in psutil [...] The problem with this approach is that it seems to report results that are slightly different in some cases. Notably it seems to differ when objects have been recently liberated from the python interpreter. In the following example, orphaned arrays are liberated by the python interpreter, which is correctly seen by psutil but not by resource..."

user_18382330
Related