Error calling a Lua function from Rust: `*mut rlua::ffi::lua_State` cannot be shared between threads safely

Viewed 154

I am developing a CLI program for rendering template files using the new MiniJinja library by mitsuhiko.

The program is here: https://github.com/benwilber/temple.

I would like to be able to extend the program by allowing the user to load custom Lua scripts for things like custom filters, functions, and tests. However, I am running into Rust lifetime errors that I've not been able to solve.

Basically, I would like to be able to register a Lua function as a custom filter function. But it's showing an error when compiling. Here is the code:

https://github.com/benwilber/temple/compare/0.3.1..lua

Error:

https://gist.github.com/c649a0b240cf299d3dbbe018c24cbcdc

How can I call a Lua function from the MiniJinja add_filter function? I would prefer to try to do this in the regular/safe way. But I'm open to unsafe alternatives if required.

Thanks!

Edit: Posted the same on Reddit and users.rust-lang.org

1 Answers

Lua uses state that is not safe to use from more than one thread. A consequence of this is that LuaFunction is neither Sync or Send.

This is being enforced by this part of the error message:

help: within `LuaFunction<'_>`, the trait `Sync` is not implemented for `*mut rlua::ffi::lua_State`

In contrast a minijinja::Filter must implement Send + Sync + 'static. (See https://docs.rs/minijinja/0.5.0/minijinja/filters/trait.Filter.html)

This means we can't share LuaFunctions (or even LuaContext) between calls to the Filters.

One option is to not pass your lua state into the closures, and instead create a new lua state every call, something like this.

env.add_filter(
    "concat2",
    |_env: &Environment, s1: String, s2: String|
    -> anyhow::Result<String, minijinja::Error> {
        lua.context(|lua_ctx| {
            lua_ctx.load(include_str!("temple.lua")).exec().unwrap();
            let globals = lua_ctx.globals();
            let temple: rlua::Table = globals.get("temple").unwrap();
            let filters: rlua::Table = temple.get("_filters").unwrap();
            let concat2: rlua::Function = filters.get("concat2").unwrap();
            let res: String = concat2.call::<_, String>((s1, s2)).unwrap();
            Ok(res)
        }
    }
);

This is likely to have relatively high overhead.

Another option is to create your rlua state in one thread and communicate with it via pipes. This would look more like this:

pub fn test() {
    let mut env = minijinja::Environment::new();
    
    let (to_lua_tx, to_lua_rx) = channel::<(String,String,SyncSender<String>)>();
    
    thread::spawn(move|| {
        let lua = rlua::Lua::new();
        lua.context(move |lua_ctx| {
            lua_ctx.load("some_code").exec().unwrap();
            let globals = lua_ctx.globals();
            let temple: rlua::Table = globals.get("temple").unwrap();
            let filters: rlua::Table = temple.get("_filters").unwrap();
            let concat2: rlua::Function = filters.get("concat2").unwrap();
            while let Ok((s1,s2, channel)) = to_lua_rx.recv() {
                let res: String = concat2.call::<_, String>((s1, s2)).unwrap();
                channel.send(res).unwrap()
            }
        })
    });

    let to_lua_tx = Mutex::new(to_lua_tx);
    env.add_filter(
        "concat2",
        move |_env: &minijinja::Environment,
         s1: String,
         s2: String|
         -> anyhow::Result<String, minijinja::Error> {
            let (tx,rx) = sync_channel::<String>(0);
            to_lua_tx.lock().unwrap().send((s1,s2,tx)).unwrap();
            let res = rx.recv().unwrap();
            Ok(res)
        }
    );
}

It would even be possible to start multiple lua states this way, but would require a bit more plumbing.

DISCLAIMER: This code is all untested - however, it builds with a stubbed version of minijinja and rlua in the playground. You probably want better error handling and might need some additional code to handle cleanly shutting down all the threads.

Related