Import a python class of non packaged module, and run in it's own thread

user_8425667 0
Viewed 96

This Stack Overflow answer details how to use importlib to import a module, and inspect to find the contained class by name.

While this works when using spec.loader.exec_module(module) to execute the module, 'MyClass' cannot be found in module when running exactly the same code in a thread, like so: threading.Thread(target=spec.loader.exec_module, args=[module])

I have set up a working example that can be run to see this effect demonstrated in a simple way, simply copy the following into two files and execute the first one.

File 1:

# test1.py

import importlib.util
import inspect
import threading

my_file = 'thread_test'
path = "thread_test.py"

spec = importlib.util.spec_from_file_location(my_file, path)
module = importlib.util.module_from_spec(spec)

# Comment out one of the following snippets to test each version:

# OPTION 1:
# Non-Threaded Version, Works:
spec.loader.exec_module(module)

# OPTION 2:
# Threaded Version, Doesn't work (Causes cls to not be found in the following loop):
# thread = threading.Thread(target=spec.loader.exec_module, args=[module])
# thread.start()

cls = None
for member in inspect.getmembers(module):
    if member[0] == 'MyClass':
        cls = member[1]

cls.color = 'blue'

cls().PrintColor()

File 2:

# thread_test.py

class MyClass:
    color = None
    
    def PrintColor(self):
        print(self.color)

Why doesn't the module contain MyClass when executed in it's own thread, yet it it does then executed in the same thread?

python-multithreading multithreading python
2 Answers

You have only started the Thread, but you have to join it and wait until it has run: thread.join(). Otherwise the module wont be imported correctly.

Full example:

import importlib.util
import inspect
import threading

my_file = 'thread_test'
path = "thread_test.py"

spec = importlib.util.spec_from_file_location(my_file, path)
module = importlib.util.module_from_spec(spec)

# Comment out one of the following snippets to test each version:

# OPTION 1:
# Non-Threaded Version, Works:
# spec.loader.exec_module(module)

# OPTION 2:
# Threaded Version, Doesn't work (Causes cls to not be found in the following loop):
thread = threading.Thread(target=spec.loader.exec_module, args=[module])
thread.start()
thread.join()

cls = None
for member in inspect.getmembers(module):
    if member[0] == 'MyClass':
        cls = member[1]

cls.color = 'blue'

cls().PrintColor()

Result:

blue

user_113559260

After discussing with others I have come to the following solution. The problem can be solved by running the loading of the module and class in it's own thread. Essentially just starting the thread sooner.

So we put it all into the following function like so:

# Run this in it's own thread
def my_function(module, class_name):
    spec.loader.exec_module(module)
    
    # Get class
    for member in inspect.getmembers(module):
        if member[0] == class_name:
            my_class = member[1]
        if my_class is None:
            my_print("Error: Could not find class " + class_name + '!')
            return

    # Set properties
    my_class.color = 'blue'
    
    # Execute code
    cls().PrintColor()

Import:

spec = importlib.util.spec_from_file_location(file, path)
module = importlib.util.module_from_spec(spec)

Then simply execute this function in it's own thread:

# Start thread
thread = threading.Thread(target=my_function, args=[module, class_name])
thread.start()
user_84256670
Related