Python: always import the last revision in the directory

user_5805827 0
Viewed 55

Imagine that we have the following Data Base structure with the data stored in python files ready to be imported:

data_base/
  foo_data/
      rev_1.py
      rev_2.py
  bar_data/
      rev_1.py
      rev_2.py
      rev_3.py

In my main script, I would like to import the last revision of the data available in the folder. For example, instead of doing this:

from data_base.foo_data.rev_2 import foofoo
from data_base.bar_data.rev_3 import barbar

I want to call a method:

import_from_db(path='data_base.foo_data', attr='foofoo', rev='last')
import_from_db(path='data_base.bar_data', attr='barbar', rev='last')

I could take a relative path to the Data Base and use glob.glob to search the last revision, but for this, I should know the path to the data_base folder, which complicates things (imagine that the parent folder of the data_base is in sys.path so the from data_base.*** import will work)

Is there an efficient way to maybe retrieve a full path knowing only part of it (data_base.foo_data)? Other ideas?

import python-3.x python
2 Answers

I think it's better to install the last version.

but going on with your flow, you may use getattr on the module:

from data_base import foo_data
i = 0
while True:
    try:
        your_module = getattr(foo_data, f'rev_{i}')
    except AttributeError:
        break
    i += 1
# Now your_module is the latest rev
user_118449250

@JohnDoriaN 's idea led me to a quite simple solution:

import os, glob

def import_from_db(import_path, attr, rev_id=None):
    """
    """
    # Get all the modules/folders names
    dir_list = import_path.split('.')
    
    # Import the last module
    exec(f"from {'.'.join(dir_list[:-1])} import {dir_list[-1]}")
    db_parent = locals()[dir_list[-1]]
    
    # Get an absolute path to corresponding to the db_parent folder
    abs_path = db_parent.__path__._path[0]
    rev_path = os.path.join(abs_path, 'rev_*.py')
    rev_names = [os.path.basename(x) for x in glob.glob(rev_path)]
    
    if rev_id is None:
        revision = rev_names[-1]
    else:
        revision = rev_names[rev_id]
    
    revision = revision.split('.')[0]
    
    # import attribute
    exec(f'from {import_path}.{revision} import {attr}', globals())

Some explanations:

  • Apparently (I didn't know this), we can import a folder as a module; this module has a __path__ attribute (found out using the built-in dir method).
  • glob.glob allows us to use regex expressions to search for a required pattern for files in the directory.
  • using exec without parameters will import only in the local namespace (namespace of the method) so without polluting the global namespace.
  • using exec with globals() allows us to import in the global namespace.
user_58058270
Related