When ruby runs out of memory for it's internal allocations, it raises NoMemoryError.
We can catch this, and even do simple operations, but it seems that if we try to continue on, even after garbage collection, we will eventually hit a fatal memory allocation problem, which on my Linux box looks like:
[FATAL] failed to allocate memory
I suspect at this point this is my OS complaining about ruby running out of memory, instead of ruby catching it.
What I don't understand is that there are many cases where we should be able to continue after we rescue a NoMemoryError (and hence give up on the allocation that caused the failure) but it seems that ruby never cleans up the mess. A simple example is to cause (and rescue) a NoMemoryError by shifting too big of a number. For example:
begin
a = 1<<10000000000000
puts "Calculated answer somehow"
rescue NoMemoryError => e
puts "RESCUE: out of memory"
end
puts "Finished first begin/rescue/end"
GC.start # This is in vain
begin
a = 1<<10000000000000
puts "Calculated answer somehow"
rescue NoMemoryError => e
puts "RESCUE: out of memory"
end
puts "Finished second begin/rescue/end"
We catch the first one, which it seems should regain the memory from what I presume to be a BigNum overflow, but that memory never comes back, not even courtesy the garbage collection. So the next time we try to start allocating a bunch of memory (in this case with the same problem) we actually hit a fatal error. The full output:
RESCUE: out of memory
Finished begin/rescue/end
[FATAL] failed to allocate memory
Is there some way to fixup the state after Ruby has hit NoMemoryError if we rescue (and hence abort) the operation that took too much memory?
It feels like this might be a problem with the Garbage Collector, possibly not properly marking memory as no longer in use when we do a rescue, which makes me think there might be a GC bug that this is revealing.