Dynamically importing Python module

user_27122 0
Viewed 9332

I have a trusted remote server that stores many custom Python modules. I can fetch them via HTTP (e.g. using urllib2.urlopen) as text/plain, but I cannot save the fetched module code to the local hard disk. How can I import the code as a fully operable Python module, including its global variables and imports?
I suppose I have to use some combination of exec and imp module's functions, but I've been unable to make it work yet.

dynamic-import python
4 Answers

Python3 version
(attempted to edit other answer but the edit que is full)

import imp

my_dynamic_module = imp.new_module("my_dynamic_module")
exec("""
class Foo:
    pass
""", my_dynamic_module.__dict__)

Foo = my_dynamic_module.Foo
foo_object = Foo()

# register it on sys
import sys
sys.modules[my_dynamic_module.__name__] = my_dynamic_module
user_43671340

I recently encountered trying to do this while trying to write unit tests for source code examples I put into a project's readme (I wanted to avoid just linking to small files or duplicating the text in a way that could get out of sync).

I came up with the following

import sys
import types
from importlib import import_module


def compile_and_install_module(module_name: str, source_code: str) -> types.ModuleType:
    """Compile source code and install it as a module.

    End result is that `import <module_name>` and `from <module_name> import ...` should work.
    """
    module = types.ModuleType(module_name, "Module created from source code")

    # Execute source in context of empty/fake module
    exec(source_code, module.__dict__)

    # Insert fake module into sys.modules. It's now a real module
    sys.modules[module_name] = module

    # Imports should work now
    return import_module(module_name)

And a quick example of how you can use it

$ cat hello.py 
def foo():
    print("Hello world")


bar = 42

$ python
Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from compile import compile_and_install_module
>>> compile_and_install_module("hello", open("hello.py").read())
<module 'hello'>
>>> import hello
>>> hello.foo()
Hello world
>>> from hello import bar
>>> bar
42

You can remove the return value and import_lib import if you

user_107785530
Related