Getting glob to follow symlinks in Python

user_600240 0
Viewed 6900

Suppose I have a subdirectory of symlinks that looks like the following:

subdir/
    folder/
        readme.txt
    symlink/ => ../hidden/
hidden/
    readme.txt

If I run the following code:

>>> from pathlib import Path
>>> list(Path('./subdir/').glob('**/readme.txt'))

I would expect the outcome to be:

subdir/folder/readme.txt
subdir/symlink/readme.txt

But the actual result is:

subdir/folder/readme.txt

I found out that this is because (for some undocumented reason) the ** operator doesn't follow symlinks.

Is there a way to change this configuration pragmatically?

symlink python-3.x python
3 Answers

pathlib.glob also doesn't work for me with ** and symlinks. I've found related issue https://bugs.python.org/issue33428.

As an alternative for Python3 you could use glob.glob with ** and recursive=True option (see details https://docs.python.org/3/library/glob.html)

In [67]: from glob import glob
In [71]: list(glob("./**/readme.txt", recursive=True))
Out[71]:
['./hidden/readme.txt',
 './subdir/folder/readme.txt',
 './subdir/symlink/readme.txt']

In [73]: list(glob("./**/readme.txt", recursive=False))
Out[73]: ['./hidden/readme.txt']

Compare to:

In [72]: list(Path('.').glob('**/readme.txt'))
Out[72]: [PosixPath('hidden/readme.txt'), PosixPath('subdir/folder/readme.txt')]
user_4183580

I've never used pathlib before, so you may extend this solution to take advantage of some of its features, but I got this to work using glob only.

from glob import glob
list(glob('./subdir/*/readme.txt'))

Output:

['./subdir/folder/readme.txt', './subdir/symlink/readme.txt']

If you're set on using glob with more than one depth of subdirectory, the hackish solution is to include variations with extra */ (e.g. ./subdir/*/*/*/readme.txt) up to some arbitrary depth, and concatenate the results from each variation.

The more appropriate way to do what you want would be to write a custom function that has the behavior you want (searches through symlinks to arbitrary depth), and handles the case of circular paths in the way you want. See this question for tips on doing this with os.walk (remember to set followlinks=True).

user_53225740

python 3.6 works for me with a call to rglob,

import pathspec

p = pathspec.Path("./subdir").rglob("readme.txt")
user_13303810
Related