Unexpected interaction between :single_use pool option and :min runners

Ouverte
#60 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
25/100
Type d'issue
Bug
Clarté
À clarifier
Activité
À l'abandon
Stack technique
elixir

Piste de recherche

No files or tests are named. Start by tracing Pool's checkout_runner/{3,4} entry points and the runner's down message through the Terminator, Pool, and Runner components. Compare the proposed scaling transitions and verify that enforcing :min with :single_use does not create a recycling loop involving idle_shutdown_after.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Runners in a pool which uses the :single_use option will terminate after they execute a function call, however, this behavior causes the pool to scale below the configured :min value.

I took a stab at fixing it, but I encountered some tricky bits that I wasn't sure how to work around.

In order to work around it in a backwards compatible way, I think the :single_use option needs to change from a boolean with default false, to an enum with values like :all, :only_dynamic, :none with default value :none. :all means that all runners, including the initial booted ones, have the single use behavior applied. :only_dynamic means that only runners booted in response to demand over the initial runners will have the single use behavior applied.

The issue I see with this is that the Terminator now needs to know what "kind" of runner it is: one that was booted to satisfy the :min pool or one that was booted to satisfy demand. This would couple the Terminator/Pool/Runner a little closer but would allow the Terminator to decide to leave initial runners alive after even in the single use pool.

Another idea is to change when the pool queries for unmet demand. Currently, the pool only scales on the "leading edge" of demand on the pool, in the pool's checkout_runner/{3,4}. Another idea is to query the pool's desired size on the "tailing edge" of the runner's lifecycle, when the parent gets the runner's down message. Then, when any runner is terminated due to the single use property the pool can correct itself.

The problem with that strategy is that it gets us into a cycle of recycling workers since the min workers go away due to the single use policy and we use dynamic runners to replace them. At this point, the pool's idle_shutdown_after will apply to the dynamic runners, then those go idle and die, and the pool boots new runners to replace them. This is especially brutal because in some environments where there aren't many requests keeping the workers occupied, they get into a cycle of restarting within a small interval of each other.

I've actually implemented that behavior on top of my change from #51 and its working pretty well, besides the constant recycling of workers. Additionally, it has me thinking about other transition states that we might want to support for scaling the pool. For my desired "constant number of overprovisioned workers" custom strategy it might make sense to ensure the pool is in the desired state on a constant interval.

My final observation is that the single use option could be implemented in a different way by making the callers end their function calls with System.stop. I'm curious why it was implemented as an option on the pool. I suppose such a strategy might not work for place_child in some cases.

Langage dominant
Elixir
Étoiles
1.2k
Forks
68
Métriques de merge des PR
Aucune PR mergée en 30 j

Préparer son environnement

Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de phoenixframework/flame

Toutes les issues de phoenixframework/flame

Issues similaires

Plus d'issues Elixir

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.