Why is the c function `open` ~4x slower on MacOS vs. an Ubuntu VM on the same machine?

user_1364192 0
Viewed 103

Why is MacOS ~4x slower to open files than an Ubuntu VM on the same machine here?

A MWE using similar settings to the code this behavior was discovered on

#include <stdio.h>
#include <fcntl.h>
#include <time.h>
int main()
{
    struct timespec tstart={0,0}, tend={0,0};
    clock_gettime(CLOCK_MONOTONIC, &tstart);

    int fd = open("/path/to/testfile.txt", O_RDONLY | O_CLOEXEC, S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH);

    clock_gettime(CLOCK_MONOTONIC, &tend);
    printf("%.0f µs\n", (((double)tend.tv_sec + 1.0e-9*tend.tv_nsec) - ((double)tstart.tv_sec + 1.0e-9*tstart.tv_nsec)) * 1.0e6);

   return 0;
}

MacOS 10.15.7, no SIP, no file vault, on a MacBook Pro with an SSD

51 µs
46 µs
49 µs
30 µs
46 µs

Ubuntu 20.04 VM (parallels) on the same machine

12 µs
12 µs
12 µs
13 µs
13 µs
fcntl ubuntu filesystems macos c
2 Answers

The time spent in the open function is unlikely to have anything to do with the disk or backing filesystem type; rather, it's mainly a matter of the operating system's implementation of system calls, and perhaps in particular its model of filesystem abstractions. Linux is a monolithic kernel without different privilege domains or memory spaces involved, and aims to make system calls very fast. At least originally MacOS X was built on the microkernel stuff Apple has been a fan of for decades, and if it's still anything like that, system calls are probably a lot more costly. They may even hook into antivirus software or stuff like that nowadays.

user_3798970

A Linux VM being faster than native MacOS appears to be explained by https://github.com/golang/go/issues/28739#issuecomment-1042652785

Just FWIW, this is a problem with Apple NVMe drives (T2 and M1 Macs alike). Their cache flush performance is abysmal. This affects both macOS and (native) Linux (presumably existing VM solutions fail to forward this as F_FULLSYNC if they run faster). You get about 46 IOPS of F_FULLSYNC on macOS on an M1 machine, and about the same on native Linux with fsync().

user_13641920
Related