I understand std::filesystem is not a speed demon. But compared to opendir/readdir I see execution times for std::filesystem::directory_iterator being a factor 8-10 larger (Ubuntu 18.04, clang 9.0.0). Now I wonder if I do something wrong or at least hope for some insight why it is (so much) slower.
In more detail: I'm comparing these two, almost identical functions:
int32_t GetNumberOfFilesStdFilesys(const std::filesystem::path& testDir) {
// Find all files in that testDir
int32_t numFiles = 0;
for (auto& dirent : std::filesystem::directory_iterator(testDir)) {
if (dirent.is_regular_file() || dirent.is_symlink()) {
// const std::string filename(dirent.path().filename());
numFiles++;
}
}
return numFiles;
}
int32_t GetNumberOfFilesStdC(const std::filesystem::path& testDir) {
// Find all files in that testDir
int32_t numFiles = 0;
DIR* dir = opendir(testDir.string().c_str());
for (struct dirent* entry = readdir(dir); entry != NULL; entry = readdir(dir
)) {
if (entry->d_type == DT_REG || entry->d_type == DT_LNK) {
// const std::string filename(entry->d_name);
numFiles++;
}
}
closedir(dir);
return numFiles;
}
They walk over a directory and count the regular and symbolic-link entries. I then run this in a loop 30000 times for a directory with 100 symlinks and 300 times for a directory with 10000 symlinks. Here the result:
Symlink (Number Files: 100, Iterations: 30000)
Std C lib CPU time: 443.91 msec Wall clock: 446.97 msec
std::filesystem CPU time: 3908.20 msec Wall clock: 3910.18 msec
Symlink (Number Files: 10000, Iterations: 300)
Std C lib CPU time: 455.26 msec Wall clock: 455.92 msec
std::filesystem CPU time: 5090.57 msec Wall clock: 5090.61 msec
the CPU time is measured with std::clock and the Walltime with std::chrono::high_resolution_clock.
I assume that directory_iterator is processing more data, maybe following the symlink (?) - are there any flags to prevent this?
Or is this simply the price of platform abstraction?
Thanks for any insight!