Unexpected interaction between :single_use pool option and :min runners

未关闭
  • 哪里不对？
#60 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
25/100
Issue 类型
缺陷
描述清晰度
需要澄清
活跃度
停滞
技术栈
elixir
领域
backend, distributed-systems

调研方向

No files or tests are named. Start by tracing Pool's checkout_runner/{3,4} entry points and the runner's down message through the Terminator, Pool, and Runner components. Compare the proposed scaling transitions and verify that enforcing :min with :single_use does not create a recycling loop involving idle_shutdown_after.

由索引模型根据 Issue 内容生成。

描述

Runners in a pool which uses the :single_use option will terminate after they execute a function call, however, this behavior causes the pool to scale below the configured :min value.

I took a stab at fixing it, but I encountered some tricky bits that I wasn't sure how to work around.

In order to work around it in a backwards compatible way, I think the :single_use option needs to change from a boolean with default false, to an enum with values like :all, :only_dynamic, :none with default value :none. :all means that all runners, including the initial booted ones, have the single use behavior applied. :only_dynamic means that only runners booted in response to demand over the initial runners will have the single use behavior applied.

The issue I see with this is that the Terminator now needs to know what "kind" of runner it is: one that was booted to satisfy the :min pool or one that was booted to satisfy demand. This would couple the Terminator/Pool/Runner a little closer but would allow the Terminator to decide to leave initial runners alive after even in the single use pool.

Another idea is to change when the pool queries for unmet demand. Currently, the pool only scales on the "leading edge" of demand on the pool, in the pool's checkout_runner/{3,4}. Another idea is to query the pool's desired size on the "tailing edge" of the runner's lifecycle, when the parent gets the runner's down message. Then, when any runner is terminated due to the single use property the pool can correct itself.

The problem with that strategy is that it gets us into a cycle of recycling workers since the min workers go away due to the single use policy and we use dynamic runners to replace them. At this point, the pool's idle_shutdown_after will apply to the dynamic runners, then those go idle and die, and the pool boots new runners to replace them. This is especially brutal because in some environments where there aren't many requests keeping the workers occupied, they get into a cycle of restarting within a small interval of each other.

I've actually implemented that behavior on top of my change from #51 and its working pretty well, besides the constant recycling of workers. Additionally, it has me thinking about other transition states that we might want to support for scaling the pool. For my desired "constant number of overprovisioned workers" custom strategy it might make sense to ensure the pool is in the desired state on a constant interval.

My final observation is that the single use option could be implemented in a different way by making the callers end their function calls with System.stop. I'm curious why it was implemented as an option on the pool. I suppose such a strategy might not work for place_child in some cases.

主要语言
Elixir
星标
1.2k
派生
68
30 天内没有已合并 PR

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

phoenixframework/flame 的其他 Issue

查看 phoenixframework/flame 的全部 Issue

相似的 Issue

更多 Elixir Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。