How to eval a function that use external modules in a web worker

Viewed 205

I am trying to implement a simple library to offload given tasks to a pool of web-workers. To do so, I am using the eval operator inside the worker to evaluate stringified functions that come from the main thread. To do so I am leveraging the webpack's plugin worker-plugin to automatically bundle workers inside the application.

First of all, I format the task to normalize arrow functions, class members, and standard functions

function format_task(task: string): string
{
    if(task.indexOf('=>') !== -1)
        return task;

    if(task.indexOf('function') !== -1)
        return `(function ${task.slice('function'.length).trim()})`;

   return `(function ${task})`;
}

Then, I simply eval the resulting function calling it:

self.onmessage = (event: MessageEvent) =>
{
    const {task, data} = event.data;
    const formatted_task = format_task(task);
    const runnable_task = eval(formatted_task);
    const result = await runnable_task(...data);
    self.postMessage(result);
} 

The main usage would be something like:

const pool = new WorkerPool(new Worker('./worker.ts'), 10);

const task = (a: number, b: number) => a + b; 

const worker_task = await pool.create_task(task);
const result = await worker_task.run(42, 12); 

So far, so good. The WorkerPool instance is handling a pool of 10 workers, the function gets executed in one of them, and the result gets returned to the main thread.

The problem comes in when I try to use an external dependency inside the task as

import {toUpper} from 'lodash';

const pool = new WorkerPool(new Worker('./worker.ts'), 10);

const task = (a: number, b: number) => 
{
   return toUpper('Result is ') + (a + b)); 
}

const worker_task = await pool.create_task(task);
const result = await worker_task.run(42, 12); 

In this case, I get that lodash_1.toUpper is not defined inside the worker. The only solution would be to manually import the same function also inside the worker environment (it works). However, I would like to keep the web-worker implementation as clean as possible for the final user. Something like:

const context: Worker = self as any;

(async () =>
{
    const pool_worker = new PoolWorker(context.postMessage.bind(context));
    context.onmessage = pool_worker.on_message.bind(pool_worker);
    await pool_worker.run();
})().catch(console.error);

I tried importing the needed libraries inside the main worker file but it does not work.

import {toUpper} from 'lodash';
const context: Worker = self as any;

(async () =>
{
    const pool_worker = new PoolWorker(context.postMessage.bind(context));
    context.onmessage = pool_worker.on_message.bind(pool_worker);
    await pool_worker.run();
})().catch(console.error);

It only works if I import it inside the PoolWorker class.

Can you think of a way to "pass" dependencies to the PoolWorker instance letting it importing it through webpack without writing a custom webpack loader/plugin?

Thank you in advance

0 Answers
Related