mmap's worst case memory usage when using MAP_PRIVATE vs MAP_SHARED

user_5645683 0
Viewed 58

I haven't seen this explicitly anywhere so I just wanted to clarify. This is all in the context of a single threaded program: say we have a 10GB text file when we open with mmap, using the MAP_PRIVATE option. Initially of course, I should expect to see 0GB resident memory used. Now say I modify every character in the file. Will this then require 10GB in resident memory? And if not, why not?

Now what if we did the same thing but with MAP_SHARED, what should I expect the resident memory usage to look like?

mmap memory c++
1 Answers

MAP_SHARED creates a mapping that is backed by the original file. Any changes to the data are written back to that file (assuming a read/write mapping).

MAP_PRIVATE creates a mapping that is backed by the original file for reads only. If you change bytes in the mapping, then the OS creates a new page that is occupies physical memory and is backed by swap (if any).

The impact on resident set size is not dependent on the mapping type: pages will be in your resident set if they're actively accessed (read or write). If the OS needs physical memory, then pages that are not actively accessed are dropped (if clean), or written to either the original file or swap (if dirty, and depending on mapping type).

Where the two types differ is in total commitment against physical memory and swap. A shared mapping doesn't increase this commitment, a private mapping does. If you don't have enough combined memory and swap to hold every page of the private mapping, and you write to every page, then you (or possibly some other process) will be killed by the out-of-memory daemon.

Update: what I wrote above applies to memory-mapped files. You can map an anonymous block (MAP_ANONYMOUS) with MAP_SHARED, in which case the memory is backed by swap, not a file.

user_134007290
Related