Node.js out of memory when running a large Jest test suite

Question

I’m running a large test suite with Jest (26.6.2) and react-testing-library (12.1.2) on Node.js (16.3.1).

At some point the test suite sometimes crashes with

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

The first idea is that the Node.js heap is full. Thus, I’ve set a heap size of 2GB via --max-old-space-size=2048 and also enabled GC logging (Node’s --expose-gc + Jest’s --logHeapUsage). The GC logging indicates that the heap space is probably not the problem (if my understanding is correct, the error message is at least misleading):

<--- Last few GCs --->

[206149:0x4b56570]  1113100 ms: Mark-sweep (reduce) 872.1 (1175.1) -> 871.2 (1114.6) MB, 1786.2 / 0.0 ms  (average mu = 0.018, current mu = 0.014) last resort GC in old space requested
[206149:0x4b56570]  1114841 ms: Mark-sweep (reduce) 871.2 (1114.6) -> 871.2 (1091.4) MB, 1740.9 / 0.0 ms  (average mu = 0.009, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
  1: 0xb00d90 node::Abort() [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
  2: 0xa1823b node::FatalError(char const*, char const*) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
  3: 0xcedbce v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
  4: 0xcedf47 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
  5: 0xea6105  [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
  6: 0xeb8588 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
  7: 0xe85b76 v8::internal::Factory::CodeBuilder::AllocateCode(bool) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
  8: 0xe86885 v8::internal::Factory::CodeBuilder::BuildInternal(bool) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
  9: 0xe8729e v8::internal::Factory::CodeBuilder::Build() [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
 10: 0x14d69fe v8::internal::RegExpMacroAssemblerX64::GetCode(v8::internal::Handle) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
 11: 0x11c1329 v8::internal::RegExpCompiler::Assemble(v8::internal::Isolate*, v8::internal::RegExpMacroAssembler*, v8::internal::RegExpNode*, int, v8::internal::Handle) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
 12: 0x11df6b7 v8::internal::RegExpImpl::Compile(v8::internal::Isolate*, v8::internal::Zone*, v8::internal::RegExpCompileData*, v8::base::Flags, v8::internal::Handle, v8::internal::Handle, bool, unsigned int&) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
 13: 0x11dfe60 v8::internal::RegExpImpl::CompileIrregexp(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, bool) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
 14: 0x11e0a5e v8::internal::RegExpImpl::IrregexpPrepare(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
 15: 0x11e0bd7 v8::internal::RegExpImpl::IrregexpExec(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, int, v8::internal::Handle, v8::internal::RegExp::ExecQuirks) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
 16: 0x1205da8 v8::internal::Runtime_RegExpExec(int, unsigned long*, v8::internal::Isolate*) [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]
 17: 0x15e7819  [./.gradle/nodejs/node-v16.13.1-linux-x64/bin/node]

The tests crash at different points (even different test classes!) but I always see a RegExp compilation in the JS stacktrace. Searching for the stacktrace pattern leads me to https://bugs.chromium.org/p/v8/issues/detail?id=10441#c5 which gives the hint that the V8 code_space memory region could be the problem:

V8 has limit of 128*MB for code objects. Each RegExp.exec (or String.match) compiles an expression and thus creates a code object, which may be large depending on the RegExp.

Indeed, when I add logging of the V8 heap space statistics via v8.getHeapSpaceStatistics(), I can see that the code_space region could be causing the crashes. The last output I see before the crash is:

Heap statistics: {
    "total_heap_size": 1251897344,
    "total_heap_size_executable": 133742592,
    "total_physical_size": 1249687440,
    "total_available_size": 1200419000,
    "used_heap_size": 968511640,
    "heap_size_limit": 2197815296,
    "malloced_memory": 4202568,
    "peak_malloced_memory": 12083656,
    "does_zap_garbage": 0,
    "number_of_native_contexts": 3,
    "number_of_detached_contexts": 0
}
Heap space statistics: [
    […]
    {
        "space_name": "old_space",
        "space_size": 1011437568,
        "space_used_size": 766010912,
        "space_available_size": 226275256,
        "physical_space_size": 1011428824
    },
    {
        "space_name": "code_space",
        "space_size": 133275648,
        "space_used_size": 124423840,
        "space_available_size": 512352,
        "physical_space_size": 131074880
    },
    […]
]

I’m wondering if the crash suggests that the test suite is just too large or if it is rather a memory leak?

Is there a way to increase the code_space region size in V8?

I’m appreciating any hint to narrow down the root cause for those crashes.

Node.js out of memory when running a large Jest test suite

0 Answers