When django load data to memory?

user_10985982 0
Viewed 288

Lets have simple model:

class NewModel(models.Model):
    json_data  = models.JSONField(default=dict)

Then lets get it:

model = NewModel.objects.get(pk=1)  # 1

data = model.json_data  # 2

value = data.get("key")  # 3

Because json_data is pretty big (about 1-2MB per value, and min. 20 values) question is, when django load data to memory?

If I understand correctly then 1 only return point to model but not load it. So it load on attribute call? Or when .get is called?

I want to understand how it works and optimise loading, so I will not load values that are unnecessary.

django-models django
1 Answers

Because json_data is pretty big (about 1-2MB per value, and min. 20 values) question is, when django load data to memory?

By default it loads all columns of the table in memory when the queryset is evaluated. .get(…) [Django-doc] is evaluated, so when you perform a NewModel.objects.get(pk=1), the column is loaded in memory.

If I understand correctly then 1 only return point to model but not load it.

No, it loads all columns of the table when it retrieves a record, to prevent a lot of extra roundtrips to the database.

I want to understand how it works and optimise loading, so I will not load values that are unnecessary.

You can use .defer(…) [Django-doc] to prevent loading columns immediately, so:

obj = NewModel.objects.defer('json_data').get(pk=1)

will not load the json_data directly. Only in case you later need the data, it will make an extra query to fetch that into memory. If you thus seldomly need json_data, that it can be worth to defer the column, of course if you need this field often, it can have a negative impact, since you increase the numbe rof roundtrips to the database.

user_675790
Related