I want to measure how much memory Django queryset use.
For example, I try the simple way.
import psutil
process = psutil.Process(os.getpid())
s = process.memory_info().rss # in bytes
for i in queryset:
pass
e = process.memory_info().rss # in bytes
print('queryset memory: %s' % (e-s))
Since iterating queryset, Django will hit a database and the result will be cached and by getting memory usage for the Python process, I try to measure queryset memory usage.
I wonder if the access would be right or there is any way to measure my goal, you guys know.
This measure is to predict if there would be any issue when trying to get a massive query result and if there is, from how many rows it results in an issue.
I know If I want to avoid caching queryset result, I can use iterator().
However, iterator() also should determine chunk_size parameter to reduce the number of hitting database and memory usage will be different depending on chunk_size.
import psutil
process = psutil.Process(os.getpid())
s = process.memory_info().rss # in bytes
for i in queryset.iterator(chunk_size=10000):
pass
e = process.memory_info().rss # in bytes
print('queryset memory: %s' % (e-s))