How can I lazily import a module in python?

user_2883245 0
Viewed 166

I have classes which require dependencies in order to be instantiated but are otherwise optional. I'd like to lazily import the dependencies and fail to instantiate the class if they aren't available. Note that these dependencies are not required at the package level (otherwise they'd be mandatory via setuptools). I currently have something like this:

class Foo:
    def __init__(self):
        try:
            import module
        except ImportError:
            raise ModuleNotFoundError("...")

    def foo(self):
        import module

Because this try/except pattern is common, I'd like to abstract it into a lazy importer. Ideally if module is available, I won't need to import it again in Foo.foo so I'd like module to be available once it's been imported in __init__. I've tried the following, which populates globals() and fails to instantiate the class if numpy isn't available, but it pollutes the global namespace.

def lazy_import(name, as_=None):
    # Doesn't handle error_msg well yet
    import importlib
    mod = importlib.import_module(name)
    if as_ is not None:
        name = as_
    # yuck...
    globals()[name] = mod

class NeedsNumpyFoo:
    def __init__(self):
        lazy_import("numpy", as_="np")

    def foo(self):
        return np.array([1,2,])

I could instantiate the module outside the class and point to the imported module if import doesn't fail, but that is the same as the globals() approach. Alternatively lazy_import could return the mod and I could call it whenever the module is needed, but this is tantamount to just importing it everywhere as before.

Is there a better way to handle this?

python
2 Answers

Pandas actually has a function import_optional_dependency which may make a good example (link GitHub) as used in SQLAlchemyEngine (link GitHub)

However, this is only used during class __init__ to get a meaningful error (raise ImportError(...) by default!) or warn about absence or old dependencies (which is likely a more practical use of it, as older or newer dependencies may import correctly anywhere if they exist, but not work or be explicitly tested against or even be an accidental local import)

I'd consider doing similarly, and either not bother to have special handling or only do it in the __init__ (and then perhaps only for a few cases where you're interested in the version, etc.) and otherwise simply import where needed

class Foo():
    def __init__(self, ...):
        import bar  # only tests for existence

    def usebar(self, value):
        import bar
        bar.baz(value)

Plausibly you could assign to a property of the class, but this may cause some trouble or confusion (as the import should already be available in globals once imported)

class Foo():
    def __init__(self, ...):
        import bar
        self.bar = bar

    def usebar(self, value):
        self.bar.baz(value)
user_45410450

Gave it a quick test with a wrapper, seems to work fine:

def requires_math(fn):
    def wrapper(*args, **kwargs):
        global math
        try:
            math
        except NameError:
            import math
        return fn(*args, **kwargs)
    return wrapper

@requires_math
def func():
    return math.ceil(5.5)

print(func())

Edit: More advanced one that works with any module, and ensures it is a module in case it's been set to something else.

from types import ModuleType

def requires_import(*mods):
    def decorator(fn):
        def wrapper(*args, **kwargs):
            for mod in mods:
                if mod not in globals() or not isinstance(globals()[mod], ModuleType):
                    globals()[mod] = __import__(mod)
            return fn(*args, **kwargs)
        return wrapper
    return decorator

@requires_import('math', 'random')
def func():
    return math.ceil(random.uniform(0, 10))

print(func())
user_24030000
Related