I would like to find files whose whole name (relative, although absolute is nice too) matches a given regular expression (i.e., like the glob module, but for regex matches instead of shell wildcard matches). Using find, one would do, for example:
find . -regex ./foo/\w+/bar/[0-9]+-\w+.dat
Of course, I could use find via os.system(...) or os.exec*(...), but I'm looking for a pure Python solution. The following code combining os.walk(...) with re module regular expressions is an easy Python solution. (It's not robust and misses many (not-so-corner-ish) corner-cases, but is good enough for my single-use purpose, locating specific data files for a one-time database insertion.)
import os
import re
def find(regex, top='.'):
matcher = re.compile(regex)
for dirpath, dirnames, filenames in os.walk(top):
for f in filenames:
f = os.path.relpath(os.path.join(dirpath, f), top)
if matcher.match(f):
yield f
if __name__=="__main__":
top = "."
regex = "foo/\w+/bar/\d+-\w+.dat"
for f in find(regex, top):
print f
But this is inefficient. Subtrees whose contents cannot match the regex (e.g., ./foo/\w+/baz/, to continue the example from above) are unnecessarily walked. Ideally, these subtrees should be pruned from the walk; any sub-directory whose path name is not a partial match for the regex should not be traversed. (I would guess that GNU find implements such an optimization, but I have not confirmed this through tests or source-code perusal.)
Does anyone know of a Python implementation of a robust regex-based find, ideally with subtree-pruning optimization? I'm hoping that I'm just missing a method in the os.path module or some third-party module.