Eloquent Model save() run out of memory

Question

I use eloquent model to do some complex migration of database and run out of memory during the processing. Can someone explain what's the reason? Thank you!

Laravel version: "v8.52.0"

Test code:

public function handle()
{
  for ($i = 0; $i < 100; $i++) {
    Customer::chunkById(1000, function ($customers) use ($i) {
      $this->print_progress();
      foreach ($customers as $customer) {
        $customer->first_name = (string)$i;
        $customer->save();
      }
    });
  }
}

Output (memory usage):

usage:    27MB - peek:    27MB
usage:    33MB - peek:    33MB
usage:    39MB - peek:    39MB
...
...
usage:   491MB - peek:   491MB
usage:   496MB - peek:   496MB
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /home/vagrant/code/billing/vendor/laravel/framework/src/Illumi                                                                                         nate/Support/Str.php on line 855
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 32768 bytes) in /home/vagrant/code/billing/vendor/symfony/error-handler/Error/                                                                                         FatalError.php on line 1```

Update: Memory leak is caused by telescope. When turn off telescope, no memroy leak occurs.

user_3330212 · Answer

You have a memory leak somewhere.

I'll assume the problem is not within the print_progress function but please, double check it (or edit your question with its content).

It's hard to give you an accurate answer since there can be many things that cause memory leaks, but try to use saveQuietly instead of save. Model events will not be dispatched and it might be the cause of your problem.

Also, check you are not using Laravel Telescope and if you do, disable it during these tests.

user_1998801 · Answer

First of all, I see your code is really strange. I understand if you are just testing stuff.

So, your logic is this:

Iterate the next step 100 times.
Each iteration will go to the database and get a collection of 1000 Customers in chunks (but you will always iterate over all your Customers).
You are going to iterate over each chunk and say the first name of each 1000 you got in the previous step is going to be the current $i index (as string). This will happen for all your Customers, but you are doing it in in chunks.

So:

You are not filtering your query in any way, so maybe chunkById is being less performant than a normal chunk, so first of all try to use that.
If you are still running out of memory, just reduce your 1000 to 500 or, personal recommendation 200 or 100, never have more than that per chunk... Getting 1000 models in a collection is not very performant wise.
You can have a code a little bit more readable or Laravel friendly by using Higher Order Messages:

public function handle()
{
    for ($i = 0; $i < 100; $i++) {
        Customer::chunk(100, function ($customers) use ($i) {
            $this->print_progress();
            
            $customers->each->update(['first_name' => (string)$i]);
        });
    }
}

But, if you want to be 100% performant, you can disregard chunk and directly update the table so that will be nearly instant compared to letting PHP do that work:

public function handle()
{
    for ($i = 0; $i < 100; $i++) {
        Customer::update(['first_name' => $i]);
        
        $this->print_progress();
    }
}

But I am not sure if you are just testing performance or what, so maybe this last code is of no use for you.

Eloquent Model save() run out of memory

2 Answers