Execute asyncronus function from shared library in a new thread (Rust)

user_10890264 0
Viewed 560

I was following Michael-F-Bryan's Dynamic Loading & Plugins chapter from his Rust FFI guide, but I am storing the plugins in a HashMap (HashMap<&'static str, Box<dyn Plugin>) instead of a Vec so that I can call the functions of a Plugin individually.

I would like the plugins to define a asynchronous function with it's own loop that communicates with the main part of the application using channels (std or tokio).

Working around the fact that you can't have async functions in traits was easy thanks to the async_trait crate, but the issue that I am now facing is, that I cannot spawn a new thread with the module because the new thread might outlive the PluginManager.

I tried to recreate this in a rust playground without the dynamic modules and I'm facing the same error here (note: this does not include any kind of channel communication)

Unlike the first error, I was unable to recreate a rust playground for the second one (as it happens at runtime). Instead of spawning a new thread, I was using tokio's event loop to handle the async functions. This works in the sandbox, but not when using a shared library as plugin. At runtime it throws:

thread '<unnamed>' panicked at 'there is no timer running, must be called from the context of Tokio runtime', /home/admin/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.0.1/src/time/driver/handle.rs:50:18

If someone knows a solution around this problem, or even had the same issue, it would be great if you could share them with me as I've already tried so much but nothing worked in my favor.

rust-tokio rust asynchronous plugins multithreading
1 Answers

You've already recognized the problem: a reference to an object owned by the PluginManager is being moved into a future that is being spawned, and thus may run on another thread than the one which owns the PluginManager. The compiler cannot know that the future will not outlive the PluginManager.

There are a number of possible solutions to this, for example:

  • Store the plugin instances inside Arc<Box<T>>, so that they are reference counted in runtime. Then even if the plugin manager did not live as long as the futures you are spawning, it would not matter

  • Make the PluginManager an immutable static singleton

In this case I suspect you want the PluginManager to be a singleton. You can do that by replacing the new constructor with a get method that lazily constructs the instance and returns a reference to it:

use once_cell::sync::Lazy;

impl PluginManager {

    pub fn get() -> &'static Self {
        static PLUGINMANAGER: Lazy<PluginManager> = Lazy::new(|| {
            let mut mgr = PluginManager {
                plugins: HashMap::new()
            };
            mgr.load_plugins();
            mgr
        });
        &PLUGINMANAGER
    }
// ...
}

Note that this lazy constructor fully initializes the static instance, including loading the plugins.

It uses once_cell::sync::Lazy from the once_cell crate. This functionality is also available in nightly std, so it will eventually become part of the standard library.

The spawn_plugins function needs to be altered slightly to specify that it requires a static self:

pub async fn spawn_plugins(&'static self) {
//...

and finally your main function becomes:

async fn main() {
    let mgr = PluginManager::get();
    mgr.spawn_plugins().await;
}

Playground Link

Regarding your other problem, that's a separate issue - stack overflow works best if you post one problem at a time.

user_14114570
Related