How to iterate over files in specific directories?

user_4443378 0
Viewed 102

I'd like to iterate over files in two folders in a directory only, and ignore any other files/directories. e.g in path: "dirA/subdirA/folder1" and "dirA/subdirA/folder2"

I tried passing both to pathlib as:

root_dir_A = "dirA/subdirA/folder1"
root_dir_B = "dirA/subdirA/folder2"
for file in Path(root_dir_A,root_dir_B).glob('**/*.json'):
    json_data = open(file, encoding="utf8")
    ...

But it only iterates over the 2nd path in Path(root_dir_A,root_dir_B).

pathlib python
2 Answers

You can't pass two separate directories to Path(). You'll need to loop over them.

for dirpath in (root_dir_A, root_dir_B):
    for file in Path(dirpath).glob('**/*.json'):
         ...

According to the documentation, Path("foo", "bar") should produce "foo/bar"; but it seems to actually use only the second path segment if it is absolute. Either way, it doesn't do what you seemed to hope it would.

user_8741880

Please check the output of Path(root_dir_A,root_dir_B) to see if it returns what you want.

In your specific case this should work:

path_root = Path('dirA')

for path in path_root.glob('subdirA/folder[12]/*/*.json'):
    ...

If your paths aren't homogeneous enough you might have to chain generators. I. e.:

from itertools import chain

content_dir_A = Path(root_dir_A).glob('**/*.json')
content_dir_B = Path(root_dir_B).glob('**/*.json')

content_all = chain(content_dir_A, content_dir_B)

for path in content_all:
    ...
user_136825590
Related