josdejong/workerpool

Concurrent async tasks on a single worker

Open

#371 创建于 2023年1月13日

在 GitHub 查看
 (1 评论) (0 反应) (0 负责人)JavaScript (164 fork)github user discovery
enhancementhelp wanted

仓库指标

Star
 (2,297 star)
PR 合并指标
 (PR 指标待抓取)

描述

Workerpool allows each thread to process only a single task at a time, which means that each task must be strictly CPU-bound, and any IO makes the pool work inefficiently.

At the same time, there are some traits of supporting multiple parallel tasks per threads in the code: https://github.com/josdejong/workerpool/blob/master/src/WorkerHandler.js#L332 - I think this is for the case when the thread is in "wait for ready" state only though, and due to this check: https://github.com/josdejong/workerpool/blob/master/src/Pool.js#L248 - it never schedules more than 1 task to a "busy" thread.

Piscina has a feature of scheduling multiple tasks to the same thread.

Are there any plans to implement the similar thing in workerpool (considering that the most of the infrastructure for this is already in the code)?

Repro of the serialization:

cat <<'EOT' > main.js
const workerpool = require("workerpool");
async function main() {
  const pool = workerpool.pool(`${__dirname}/worker.js`, { minWorkers: 1, maxWorkers: 1 });
  await Promise.all([
    pool.exec("test", [1]),
    pool.exec("test", [2]),
    pool.exec("test", [3]),
    pool.exec("test", [4])
  ]);
}
main();
EOT

cat <<'EOT' > worker.js
const workerpool = require("workerpool");
const delay = require("delay");
workerpool.worker({
  test: async (v) => {
    console.log(`[${v}] started`);
    await delay(1000);
    console.log(`[${v}] finished`);
  }
});
EOT

node main.js

[1] started
[1] finished
[2] started
[2] finished
[3] started
[3] finished
[4] started
[4] finished

贡献者指南