In node.js, what is the difference between memory allocated in the node.js heap vs. memory reported as "external" as seen in process.memoryUsage()? As there advantages or disadvantages to one over the other? If you were going to have a particularly large object (gigabytes), does it matter which one is used? That doc only says this:
external refers to the memory usage of C++ objects bound to JavaScript objects managed by V8.
It doesn't tell you what any consequences or tradeoffs might be of doing allocations that use this "external" memory instead of the heap memory.
I'm examining tradeoffs between using a large Array (regular Javascript Array object) vs. a large Uint32Array object. Apparently, an Array is always allocated in the heap, but a Uint32Array is always allocated from "external" memory (not in the heap).
Here the background. I have an object that needs a fairly large array of 32 bit integers as part of what it does. This array can be as much as a billion units long. I realized that I could cut the storage occupied by the array in half if I switched to a Uint32Array instead of a regular Array because a regular Array stores things as double precision floats (64 bits long) vs. a Uint32Array stores things as 32 bit values.
In a test jig, I measured that the Uint32Array does indeed use 1/2 the total storage for the same number of units long (32-bits per unit instead of 64-bits per unit). But, when looking at the results of process.memoryUsage(), I noticed that the Uint32Array storage is in the "external" bucket of storage whereas the array is in the "heapUsed" bucket of storage.
To add to the context a bit more, because a Uint32Array is not resizable, when I do need to change its size, I have to allocate a new Uint32Array and then copy data over from the original (which I suppose could lead to more memory fragmentation opportunites). With an Array, you can just change the size and the JS engine takes care of whatever has to happen internally to adjust to the new size.
What are the advantages/disadvantages of making a large storage allocation that's "external" vs. in the "heap"? Any reason to care?