why is readdir not multi thread safe?

liuli_1997 0
Viewed 64

Thanks for reading this question, I only have basic C knowledge and I am not familiar with linux system. Here is what i know about readdir：

  1. someone said that readdir() is not multi thread safe beacuse it uses static varible,so that readdir can be described as this
    struct dirent* readdir() {
     static struct dirent; //read action 
     return &dirent;
    }
  1. However, the actual glibc implementation is multi thread safe when you use readdir() to different dirstream (which is DIR struct), but not safe when you read the same dirstream (https://lwn.net/Articles/696474/)
  2. readdir_r has been deprecated since glibc 2.24. Here is the reason
    https://man7.org/linux/man-pages/man3/readdir_r.3.html

My question:

  1. i did not see any static varible in glibc source code, where is the static varible?
    Here is the source code I read. I can only see that it uses lock to ensure multi thread safe
    https://github.com/lattera/glibc/blob/master/sysdeps/posix/readdir.c
  2. I try to use multi thread to read same dirstream such as:
    DIR* des_dir=opendir(xxx);
    pthread_create(pid,0,custom_readdir,des_dir);
    Each thread shares the same dirstream, and the result is OK. Each thread outputs part of the directory correctly, so how do I understand "In cases where multiple threads read from the same directory stream unsafe?"
  3. If readdir() is indeed unsafe, how can I read a directory in a safe way? I did not find a function which can replace readdir().
glibc linux c
2 Answers

i did not see any static varible in glibc source code, where is the static varible?

There is no static variable. The code you pointed to uses a buffer in the directory stream to hold and return struct dirent objects. So it is unsafe to use multiple threads with the same directory stream because they could conflict over use of the single buffer.

I try to use multi thread to read same dirstream such as: DIR* des_dir=opendir(xxx); pthread_create(pid,0,custom_readdir,des_dir); Each thread shares the same dirstream, and the result is OK. Each thread outputs part of the directory correctly,…

You got unlucky in your experiment; the execution of the threads happened to fail to conflict. The buffer used does have room for multiple struct dirent, allowing some opportunity for multiple threads to use different parts, making observation of a failure less likely, especially when testing with only a few directories.

If readdir() is indeed unsafe, how can I read a directory in a safe way?

Open a separate directory stream for each thread. Or use one thread to read the directory stream and have it copy the desired directory information and pass it to other threads to do the work for the directories.

Eric Postpischil0

If readdir() is indeed unsafe, how can I read a directory in a safe way?

In all POSIXy systems, you can use scandir(), scandirat(), glob(), and nftw(). nftw() is the best option for traversing entire directory trees (looking for specific files), scandir()/scandirat() for scanning individual directories.

Of these, scandir() and scandirat() are thread-safe; and nftw() is thread-safe except wrt. current working directory during traversal.

In Linux and BSDs, you can also use the <fts.h> functions.

userWm7k10
Related