I am trying to implement a simple library to offload given tasks to a pool of web-workers. To do so, I am using the eval operator inside the worker to evaluate stringified functions that come from the main thread. To do so I am leveraging the webpack's plugin worker-plugin to automatically bundle workers inside the application.
First of all, I format the task to normalize arrow functions, class members, and standard functions
function format_task(task: string): string
{
if(task.indexOf('=>') !== -1)
return task;
if(task.indexOf('function') !== -1)
return `(function ${task.slice('function'.length).trim()})`;
return `(function ${task})`;
}
Then, I simply eval the resulting function calling it:
self.onmessage = (event: MessageEvent) =>
{
const {task, data} = event.data;
const formatted_task = format_task(task);
const runnable_task = eval(formatted_task);
const result = await runnable_task(...data);
self.postMessage(result);
}
The main usage would be something like:
const pool = new WorkerPool(new Worker('./worker.ts'), 10);
const task = (a: number, b: number) => a + b;
const worker_task = await pool.create_task(task);
const result = await worker_task.run(42, 12);
So far, so good. The WorkerPool instance is handling a pool of 10 workers, the function gets executed in one of them, and the result gets returned to the main thread.
The problem comes in when I try to use an external dependency inside the task as
import {toUpper} from 'lodash';
const pool = new WorkerPool(new Worker('./worker.ts'), 10);
const task = (a: number, b: number) =>
{
return toUpper('Result is ') + (a + b));
}
const worker_task = await pool.create_task(task);
const result = await worker_task.run(42, 12);
In this case, I get that lodash_1.toUpper is not defined inside the worker. The only solution would be to manually import the same function also inside the worker environment (it works). However, I would like to keep the web-worker implementation as clean as possible for the final user. Something like:
const context: Worker = self as any;
(async () =>
{
const pool_worker = new PoolWorker(context.postMessage.bind(context));
context.onmessage = pool_worker.on_message.bind(pool_worker);
await pool_worker.run();
})().catch(console.error);
I tried importing the needed libraries inside the main worker file but it does not work.
import {toUpper} from 'lodash';
const context: Worker = self as any;
(async () =>
{
const pool_worker = new PoolWorker(context.postMessage.bind(context));
context.onmessage = pool_worker.on_message.bind(pool_worker);
await pool_worker.run();
})().catch(console.error);
It only works if I import it inside the PoolWorker class.
Can you think of a way to "pass" dependencies to the PoolWorker instance letting it importing it through webpack without writing a custom webpack loader/plugin?
Thank you in advance