Docker Windows - still rising RAM usage with django bulk_create()

Viewed 25

I'm importing large amounts of data to django from csv. I created script with django-extensions which is creating objects like this:

def process_row(row):
    return {"foo": row[0], "bar": row[1]}

data = []
for row csv.reader(csvfile):
    if len(data) > 5000:
        MyModel.objects.bulk_create(data)
        data = []

    data.append(process_row(row))

Basically, this works really good and fast, but the problem is still rising RAM usage by docker container. It never goes down. At the beggining container uses ~1.5GB, then after tens of bulk_create() it uses up to 10GB and I have to restart Docker Desktop to reset that.

P.S. docker stats says it only uses ~3GB, so I think the problem is with Docker Desktop

docker stats screenshot

0 Answers
Related