How to check does a file imports from another file in Python

user_13161155 0
Viewed 896

Suppose i have a project structure like this

src
└── app
    ├── main.py
    ├── db
    │   └── database.py
    ├── models
    │   ├── model_a.py
    │   └── model_b.py
    └── tests
        ├── test_x.py
        └── test_y.py

I want to check which file uses a class or a function from another file. I have a class called Test in main.py

class Test:
    pass

I used that class in model_a,

from ..main import Test

But in model_b i used

from ..main import Test
from ..db.database import Data

I want to to check which file uses another file, just like tree command, just a folder name is enough so i tried an old method but it was inefficient ,dirty and that was not something that i expect. The method was i created a file in src named check.py, i imported all packages

from app.db import database
from app.models import model_a, model_b
from app.tests import test_x, test_y
from app import main 

print('__file__={0:<35} | __name__={1:<20} | __package__={2:<20}'.format(__file__,__name__,str(__package__)))

And i added this line in the bottom of all files

print('__file__={0:<35} | __name__={1:<20} | __package__={2:<20}'.format(__file__,__name__,str(__package__)))

So when i run check.py i get this result

__file__=/home/yagiz/Desktop/struct/src/app/main.py | __name__=app.main             | __package__=app                 
__file__=/home/yagiz/Desktop/struct/src/app/db/database.py | __name__=app.db.database      | __package__=app.db              
__file__=/home/yagiz/Desktop/struct/src/app/models/model_a.py | __name__=app.models.model_a   | __package__=app.models          
__file__=/home/yagiz/Desktop/struct/src/app/models/model_b.py | __name__=app.models.model_b   | __package__=app.models          
__file__=/home/yagiz/Desktop/struct/src/app/tests/test_x.py | __name__=app.tests.test_x     | __package__=app.tests           
__file__=/home/yagiz/Desktop/struct/src/app/tests/test_y.py | __name__=app.tests.test_y     | __package__=app.tests           
__file__=/home/yagiz/Desktop/struct/src/check.py | __name__=__main__             | __package__=None

The result is dirty and doesn't meet my expectations is there a way to get a output like this?

main.py = app/models/model_a, app/models/model_b   # These files imports something from main.py
models_b = None                                    # No file imports from models_b

Update, i tried @Hessam Korki's suggestion it doesn't works.

I looked up the source code of modulefinder and i found it adds a badmodule in every import statement which is not useful for me.

Here is how did it go, first i created a function, also i created an another project structure.

src
├── empty.py
├── __init__.py
├── main.py
├── module_finder.py
├── other
│   └── other.py
├── test
│   └── some_app.py
└── this_imports.py

Here is the module_finder.py that contains my function

from modulefinder import ModuleFinder
file_names = ["this_imports.py", "main.py", "test/some_app.py", "other/other.py", "empty.py"]

def check_imports(file_names):
    finder = ModuleFinder()
    for file in file_names:
        finder.run_script(file)
        print("\n", file)
        for name, mod in finder.modules.items():
            print('%s: ' % name, end='')
            print(','.join(list(mod.globalnames.keys())[:3]))

            print('\n'.join(finder.badmodules.keys()))

Empty file is empty(as expected), in main.py i have

class Test:
    pass

In this_imports.py i only have

from src.main import Test

In other/other.py i have

from src.main import Test 
from src.test import DifferentTest

And for the last one in test/some_app.py i have

from src.main import Test

class DifferentTest:
    pass

So the result should be:

empty.py = None
main.py = None
other/other.py = src.main , src.test
test/some_app.py = src.main
this_imports.py = src.main

But the function gives a wrong result, here is the output:

 Filename:  this_imports.py
__main__: Test
src.main

 Filename:  main.py
__main__: Test,__module__,__qualname__
src.main

 Filename:  test/some_app.py
__main__: Test,__module__,__qualname__
src.main

 Filename:  other/other.py
__main__: Test,__module__,__qualname__
src.main
src.test

 Filename:  empty.py
__main__: Test,__module__,__qualname__
src.main
src.test
import python
2 Answers

I believe python's Modulefinder will effectively solve your problem. There is a key named '__main__' in the Modulefinder().items() which holds the modules that were imported in a python file. After running the script through your project and storing the data in a way that suits your purpose, you should be good to go

user_136052640

What you are looking for is to find import dependencies in your package modules. You can run a static analysis on your package directory and parse the import nodes in the syntax trees (ast), and build a dependency graph. Something like below:

import os
from ast import NodeVisitor, parse
import networkx as nx

class Dependency():
    def __init__(self, root):
        self.root = root
        self.base = os.path.basename(root)
        self.dependency = nx.DiGraph()
        self.visitor = NodeVisitor()
        self.visitor.visit_ImportFrom = self.visit_ImportFrom
        
        self.current_node = None
        self.dependency.add_node = self.base
        
    def visit_ImportFrom(self, node):
        self.dependency.add_edge(node.module, self.current_node)
        self.visitor.generic_visit(node)
        
    def run(self):
        for root, dirs, files in os.walk(self.root):
            for file in files:
                full_path = os.path.join(root+os.sep, file)
                loc = full_path.split(self.root+os.sep)[1].replace(os.sep,'.')
                self.current_node = self.base+'.'+loc
                with open(full_path) as fp:
                    src = fp.read()
                    tree = parse(src)
                    self.visitor.generic_visit(tree)
                    
        dependency = {}
        for src, target in nx.dfs_edges(self.dependency):
            if src in dependency:
                dependency[src].add(target)
            else:
                dependency[src] = set([target])
                
        return dependency

For the root location of any package you want to map the import dependencies, you need to do the following then:

root = "path/to/your/src"
d = Dependency(root)
d.run()

This will return the dependency tree (as a dict). Note, we parsed only ImportFrom, you need to add Import to make it complete. Also, all imports are assumed absolute here (i.e. no .. etc). If required, you can add that too (check the level field of the ImportFrom node to do that).

user_47496280
Related