Retrieving the source code dependencies of a python 3 function

Viewed 91

Using the AST in python 3, how do you build a directory or list of code dependencies of a given function?

Consider the following code, where my_clever_function has the desired behaviour:

////// myfile2.py

import numpy as np

a = 1
a += 1

def my_other_function():
    def f():
        return a
    return np.random.randint() + f()


////// myfile1.py
import numpy as np
from . myfile2 import my_other_function

def external(a, b):
    return np.sqrt(a * b) + my_other_function

class A:
    def afunc(self, a, b):
        v = external(a, b)
        return v


>>> my_clever_function(A.afunc)
[myfile1.A.afunc, myfile1.external, myfile2.my_other_function, myfile2.a]

with the following structure:

project/
    myfile1.py
    myfile2.py

I want to retrieve the dependencies of the method afunc as a list. I'm assuming that there is no funny business about functions altering global variables.

  1. external is a dependency because it is not defined inside A.afunc
  2. np.sqrt is not a "dependency" (in this sense anyway) because it is not defined in my project
  3. likewise for np.random.randint
  4. my_other_function is a dependency because it is not defined inside A.afunc
  5. f is not a dependency because it is inside my_other_function
  6. f needs the global variable a.

My motivation is to see if there have been any code changes between two project versions (in git perhaps). We could find the dependencies of function like above and store the source. In the future, we find the dependencies again and see if the source code is different. We only compare the bits that are required (barring any funny global variables messing inside functions).

It is possible to walk the AST with python's builtin module ast. So my_clever_function could look like this:

import ast
import dill

class Analyzer(ast.NodeVisitor):
    def __init__(self):
        self.stats = {...}

    ...

    def report(self):
        pprint(self.stats)

def my_clever_function(f):
    source = dill.source.getsource(f)
    tree = ast.parse(source)
    analyser = Analyser()
    analyser.visit(tree)

But how do you walk from a given function outwards to its dependencies? I can see how you can just list symbols (https://www.mattlayman.com/blog/2018/decipher-python-ast/) but how do only list those which depend on the start node?

0 Answers
Related