Python import from byte IO (or more generally from an object in memory)

user_7496549 0
Viewed 867

Context:

I currently have a program which loads a set of plugins from their file paths (in a mapped network drive) using the method shown in another SO thread. These plugins are designed to be rolling release, which means I need constant access to writing them. The current mechanism locks the files so I have to ask everyone to close the software so I can update the files.

The question:

I was wondering if there was a way to, possibly using a similar method to that linked above, import a file from an io.BytesIO object of the plugin's raw contents (hence unlocking the file for me to make changes as I please).

More generally:

More specifically, can I keep the raw module contents in memory without touching a physical disk? If such a thing is not possible, is there a way to fully load these modules into memory so I can then unlock the files being imported?

As I have stated in my comment, I understand you can mount a virtual-filesystem on a Linux-based OS (which could have solved my problem), though sadly I developing for Windows and Microsoft can never make your life easy! :-)

Note:

I am not asking where I can copy these files to import them from a local version (e.g. temp, cache, etc.).

I understand this is quite a specialist question so any help is much appreciated

python-importlib bytesio python-import python
1 Answers

While not being from an io.BytesIO object as I originally asked for, I was able to import a module from its source after finding this incredibly helpful article. I have not copied the code here as it is quite large, though I was able to get it to successfully import the virtual module.

The following code is after I modified the loader to remove the common prefix, and creates a class of the module by first executing the source, getting the globals from it and finally using Python's type method to create the module class.

It is not particularly pretty and definitely breaks some Python style recommendations, so I am definitely open to improvements!

source = """def hello():
    print("I don't want to say hi to the world")"""
name = "my_module"

glo = {}
exec(source, glo)

injector = DependencyInjector()
injector.provide(name, type(name, (), glo))
injector.install()

foo = __import__(name)
foo.hello()
user_74965490
Related