How to map physical memory with mmap?

Should I use memory-mapped files aggressively instead of heap on server applications?

• My server application typically deals with transient memory allocations of blobs (binary objects) ranging from 1 MB - 300 MB. These blobs are uncompressed, processed (modified), then re-compressed. The entire process took less than a fraction of a second on modern CPUs. I also have a phobia of a condition called "out of memory". My rationale is this: when the application process runs out of memory, there is a short moment of time where every thread and every module will see that new memory allocations fail. My application depends on some third-party modules and source code of various level of robustness, so it's not guaranteed that all modules/threads could cleanly back-track from an allocation failure. The way I deal with memory-mapped file is that I only map in the section of memory that I'm currently using. I see it as a hint to the OS that when I unmap a section of the file, that I'm not going to use that section for the next moment, so the OS could page that out if it wish to. Is my understanding correct? Or was it wishful-thinking and just an abuse of the OS memory system?

• Answer:

  Instead of mmap/munmap, you can use mlock/munlock on the required portions of the file. munmap will move the pages to Inactive - by decreasing reference count - thus making them first candidates to be gobbled up during memory pressure. If you use mlock/munlock, pages are still active. So they are not the immediate candidates for reclamation during memory pressure. This may help if you application needs to revisit those portions again in near future.