Reliably getting the calling function's file path

Viewed 367

Say I have a function foo(). When invoked, I want that function to reliably determine the file path of the module (or the calling function) that invoked it. The issue is that various functions in the inspect module only return relative paths (relative to the working directory at the time of import, I assume). However, my code switches working directories multiple times and I don't know the working directory in question.

import inspect
import os

def foo():
    frame = inspect.stack()[0]
    print(frame.filename)
    print(os.path.abspath(frame.filename))

foo()

os.chdir("/")
print("New CWD:", os.getcwd())

foo()

Output:

$ cd ~
$ python3 foo.py
foo.py
/home/user/foo.py
New CWD: /
foo.py
/foo.py

Through auto-completion in my IDE I discovered that there's an undocumented function inspect.getabsfile(). Unfortunately, that doesn't work reliably, either:

import inspect

def foo():
    print("Hello")

print(inspect.getabsfile(foo))

os.chdir("/")
print("New CWD:", os.getcwd())

print(inspect.getabsfile(foo))

Output:

$ cd ~
$ python3 foo.py
/home/user/foo.py
New CWD: /
/foo.py

If I understand this bug report correctly, __file__ and code.co_filename (docs) were (are?) going to become absolute paths at some point that hasn't happened (yet). What can I do to solve this issue in the meantime?

Update: Turns out the main issue with the above two example scripts is that they access objects / stack frames originating in the main module. As soon as one imports an object from a different module or considers the stack frame of a function living in another module, it works. In a similar way, in other modules than __main__, __file__ indeed gives absolute path. (See also this issue in the Python bug tracker.) I'm at a loss as to why the main module is being treated differently, though.

1 Answers

With use of closure technique, you can store script path in import time, thus just checking if a function is from __main__ is enough to get path of all 3 following cases:

  • from ~ import ~
  • import ~
  • object defined in main script

Following example demonstrates all 3:

from sys import modules
from os.path import abspath
from os import getcwd, chdir

from Module import a
import timeit


def closure():
    original_working_dir = abspath(__file__)  # store absolute path for linux

    def _get_path(obj):
        nonlocal original_working_dir

        try:
            return abspath(obj.__file__)

        except AttributeError:  # it's probably not a module
            if obj.__module__ == '__main__':  # check if module is same as local:
                return abspath(original_working_dir)

            return abspath(modules[obj.__module__].__file__)
    return _get_path


get_path = closure()


def test_func():
    pass


def test_output():
    print(f"\nworking directory: {getcwd()}")
    print(f"import from: {get_path(a)}")
    print(f"simple import: {get_path(timeit)}")
    print(f"local function: {get_path(test_func)}")


if __name__ == '__main__':
    test_output()
    chdir('/')
    test_output()

output for windows:

working directory: Z:\github\StackOverFlow\63865883
import from: Z:\github\StackOverFlow\63865883\Module.py
simple import: C:\Users\--\AppData\Local\Programs\Python\Python38\lib\timeit.py
local function: Z:\github\StackOverFlow\63865883\63865883.py

working directory: Z:\
import from: Z:\github\StackOverFlow\63865883\Module.py
simple import: C:\Users\--\AppData\Local\Programs\Python\Python38\lib\timeit.py
local function: Z:\github\StackOverFlow\63865883\63865883.py

Ubuntu:

working directory: /mnt/z/github/StackOverFlow/63865883
import from: /mnt/z/github/StackOverFlow/63865883/Module.py
simple import: /usr/lib/python3.8/timeit.py
/mnt/z/github/StackOverFlow/63865883/63865883.py
local function: /mnt/z/github/StackOverFlow/63865883/63865883.py

working directory: /
import from: /mnt/z/github/StackOverFlow/63865883/Module.py
simple import: /usr/lib/python3.8/timeit.py
/mnt/z/github/StackOverFlow/63865883/63865883.py
local function: /mnt/z/github/StackOverFlow/63865883/63865883.py

Main reason why your code worked correctly time-to-time is because __file__ attribute of main script depends on whether you ran python script with relative path or absolute path.

Seems like __file__ for main script stores file parameter given to python interpreter from terminal.

For example, when running this file:

print(__file__)

when called relatively via python3 scratch.py:

scratch.py

You get absolute path for it.

with absolute path python3 /mnt/c/~~/scratches/scratch.py:

/mnt/c/Users/--/AppData/Roaming/JetBrains/PyCharm2020.2/scratches/scratch.py

You get full path for __file__.

Related