Performance of Win32 memory mapped files vs. CRT fopen/fread

user_343841 0
Viewed 9301

I need to read (scan) a file sequentially and process its content. File size can be anything from very small (some KB) to very large (some GB).

I tried two techniques using VC10/VS2010 on Windows 7 64-bit:

  1. Win32 memory mapped files (i.e. CreateFile, CreateFileMapping, MapViewOfFile, etc.)
  2. fopen and fread from CRT.

I thought that memory mapped file technique could be faster than CRT functions, but some tests showed that the speed is almost the same in both cases.

The following C++ statements are used for MMF:

HANDLE hFile = CreateFile(
    filename,
    GENERIC_READ,
    FILE_SHARE_READ,
    NULL,
    OPEN_EXISTING,
    FILE_FLAG_SEQUENTIAL_SCAN,
    NULL
    );

HANDLE hFileMapping = CreateFileMapping(
    hFile,
    NULL,
    PAGE_READONLY,
    0,
    0,
    NULL
    );

The file is read sequentially, chunk by chunk; each chunk is SYSTEM_INFO.dwAllocationGranularity in size.

Considering that speed is almost the same with MMF and CRT, I'd use CRT functions because they are simpler and multi-platform. But I'm curious: am I using the MMF technique correctly? Is it normal that MMF performance in this case of scannig file sequentially is the same as CRT one?

Thanks.

memory-mapped-files winapi c++ performance
5 Answers

Using ReadFile:

  • Enters Kernel Mode
  • Does a memcpy from the Disk Cache
  • If data isn't in the Disk Cache, triggers a Page Fault which makes the Cache Manager read data from the disk.
  • Exits Kernel Mode
  • Cost of entering and leaving Kernel Mode was about 1600 CPU cycles when I measured it.
  • Avoid small reads, since every call to ReadFile has the overhead of entering and leaving Kernel Mode.

Memory Mapped Files:

  • Basically places the Disk Cache right into your application's address space.
  • If data is in the cache, you just read it.
  • If data isn't there, triggers a Page Fault that makes the Cache Manager read data from the disk. (There is a User/Kernel mode transition to handle this exception)
  • Disk reads don't always succeed. You need to be able to handle memory exceptions from the system, otherwise a disk read failure will be an application crash.

So both ways will use the same Disk Cache, and will use the same mechanism of getting data into the cache (Page Fault exception -> Cache Manager reads data from the disk). The Cache Manager is also responsible for doing data prefetching and such, so it can read more than one page at a time. You don't get a page fault on every memory page.

So the main advantages of Memory-Mapped files are:

  • Can possibly use data in-place without copying it out first
  • Fewer User<->Kernel Mode transitions (depends on access patterns)

And the disadvantages are:

  • Need to handle access violation exceptions for failed disk reads
  • Takes up address space in the program to map entire files
user_23003960
Related