In Python 3.x, why does os.chdir() behave differently when used with os.listdir() and os.scandir() ? (Python 3.9.7)

user_3208862 0
Viewed 206

I am puzzled by the behavior of os.chdir() when called on items generated by os.listdir() versus items generated by os.scandir(). I realize that os.scandir() creates DirEntry objects, whereas os.listdir() does not (maybe just returning strings), but if that explains the behavior I'm seeing, I still don't quite get why. Here's what I mean:

I have a directory with three subdirectories. I want to get into each subdirectory, do something, then go into the next one and do the same thing, etc. For pedagogical purposes, I'm not using os.walk or anything else sophisticated. Here's the code that shows the confusing (to me) result (note that this is just test code, so I'm doing the simplest possible "thing" here, i.e., printing something):

def listdir_test(path="D:/python-venvs/pyinst"):
    print("\n----- listdir() output ---------")
    for item in os.listdir(path):
        if os.path.isdir(item):
            print(item)
            os.chdir(item) # this happens only once, for the first directory found
            print(os.getcwd())

def scandir_test(path="D:/python-venvs/pyinst"):
    print("\n-------- scandir() output ------------")
    for item in os.scandir(path):
        if os.path.isdir(item):
            print(item)
            os.chdir(item) # this happens three times, for each of the directories found
            print(os.getcwd())

listdir_test()
scandir_test()

As you can see, these functions are identical except that one operates on listdir() output, and the other operates on scandir() output. But the first function gives up after only one os.chdir(item) operation, rather than doing it for all three directories. The second function behaves as I expect, repeating the os.chdir(item) operation for each of three directories. Here is the output of the above code:

----- listdir() output ---------
1.3.1.2-modules
D:\python-venvs\pyinst\1.3.1.2-modules

-------- scandir() output ------------
<DirEntry '1.3.1.2-modules'>
D:\python-venvs\pyinst\1.3.1.2-modules
<DirEntry 'packages'>
D:\python-venvs\pyinst\packages
<DirEntry 'pyinst-venv'>
D:\python-venvs\pyinst\pyinst-venv

I do not understand why os.chdir(item) apparently causes the for loop to bail out when I'm using it with os.listdir() output. There's probably a good reason. For learning purposes, I'd love to know what it is, so I can better understand these functions.

python
1 Answers

When treated as a path (by functions like os.chdir), a DirEntry object is converted to its path attribute, not just a bare name. In contrast, os.listdir returns bare names.

In your os.listdir code, you're passing bare names around, which are treated as relative to the current working directory. Once you change the current working directory with os.chdir, these names no longer resolve to what you wanted them to resolve to.

In contrast, your os.scandir code uses file paths, and since the original scandir argument was an absolute path, the paths you get out of scandir are absolute too. The interpretation of an absolute path doesn't depend on the current working directory, so stuff doesn't break when you change the working directory.

user_23571120
Related