Reconcile the Scaler and propagate replicas to the StatefulSet
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 30/100
- Issue 类型
- 功能
- 描述清晰度
- 需要澄清
- 活跃度
- 活跃
- 技术栈
- kubernetes, rust
- 领域
- devops, infrastructure
调研方向
首先检查 operator-rs#1181,以及 commons-operator 中现有的 CRD 安装和 conversion-webhook 模式。跟踪产品 CRD 如何表示副本,以及产品 operator 如何 reconcile StatefulSet,然后与 smooth-operator 团队解决列出的 CRD、selector、ownership 和初始副本决策。完成标准是:Scaler 已安装,一个产品(例如 NiFi)已完成端到端接入,并且 reconciler 将 Scaler 状态传播到 StatefulSet,同时不添加 HPA 或 autoscaling 状态机相关工作。
由索引模型根据 Issue 内容生成。
描述
Part of #872
Problem
We added the Scaler CRD to operator-rs in 0.110.0.
Now we need to make it useful :)
Scope
This issue is not about making autoscaling fully work or even "safe".
That comes later on top of this.
The scope of this is purely:
-
Ship the CRD: commons-operator installs
Scalerthe same way it already installsAuthenticationClasset. al. including the conversion webhook stuff -
Extend
replicasin the product CRDs so a role group can declare that something else owns its replica count.
"Pseudocode":
roleGroups:
default:
replicas: 3 # unchanged
roleGroups:
default:
replicas:
externallyScaled: {} # operator creates a Scaler, something else sets the number (that bit is out of scope)
All of this is up-to-decision. I never know what we can do with complex enums and so on so I'll leave the CRD decision to the implementer.
This or something like it was proposed by @soenkeliebau in https://github.com/stackabletech/operator-rs/pull/1181.
I'm not sure how much of that design was thought through vs. AI generated.
Further tickets will require additions here for "real" autoscaling. So we need to consider a "auto" variant.
And the normal "fixed" replica count obviously also needs to continue working.
[!NOTE]
This does not contain any mention of min/max anywhere. There is a good reason for that. Don't add it. It's explained in https://github.com/stackabletech/issues/issues/881
- A reconcile function in operator-rs that a product operator calls. Reads spec.replicas from the Scaler, writes it to the StatefulSet, writes status.replicas and status.selector back. Here I'm super unsure how it fits with the smooth-operator team. So check with them if this makes sense :)
One thing Claude flagged during the review of Scaler and I don't know if there is a good reason for this:
status.selector is an Option<String> and an absent selector does not fail.
The HPA controller counts ready pods matching that selector and multiplies that number by its computed ratio so a selector matching nothing produces a desired count of zero.
If that is correct then I agree with Claude that this should be required rather than optional.
But again maybe @soenkeliebau and @Techassi had a good reason.
-
Wire up one product so this is testable end to end.
-
When that has been proven and all decisions have been finalised: Roll this out everywhere
Out of scope
- Nothing with any state machine, no guard, hook (see #872 ). This ticket only ever leaves the Scaler in Idle.
- We don't build or integrate anything that actually writes
scaler.spec.replicas-> That is for later. For testing a HPA can be used. - Any per-product drain work.
- Creating HPAs
Open questions
- Who reconciles the Scaler? Sönke's experimental PR put it in the nifi-operator. I think that makes sense. Right now there are no product-specific things we do but later we might wan tto.
- Which product first? Hive would be easier because it's stateless but more useful for customers would be NiFi. Therefore, unless there are objections do NiFi first
- CRD shape
- selector -> Optional or not?
- What replica count does the StatefulSet start at if someone sets externallyScaled and nothing ever writes to the Scaler? Does the operator seed it, and with what?
Expected effort
8 days, if it takes longer, stop and tell me.
My reasoning: I hope that most of the time would be spent on decisions and design and later testing and that it's not actually a lot of code and the code that is needed can be mostly AI generated.
The rollout follows existing patterns.
Notes
operator-rs#1181 may be wroth a look. It contains parts of this issue (outdated etc. but the ideas are partially there).
It uses StackableScaler and generates HPAs which I explicitly do NOT want us to do.
- 主要语言
- 没有语言数据
- 星标
- 2
- 派生
- 0
- PR 合并指标
- 30 天内没有已合并 PR
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
stackabletech/issues 的其他 Issue
-
难度 5/5 一周以上 新手友好度 25/100
stackabletech/issues#892 ·
-
Metadata Store 未关闭
难度 5/5 一周以上 新手友好度 25/100
stackabletech/issues#891 · 1 条评论 · 1 个 reaction ·
-
难度 2/5 1-3 小时 新手友好度 50/100
stackabletech/issues#890 ·
-
epic
stackabletech/issues#889 · 已指派 2 人 ·
-
stackabletech/issues#888 · 1 条评论 · 已指派 1 人 ·
查看 stackabletech/issues 的全部 Issue
相似的 Issue
-
Solved site promotion gate fails on runner PHP patch drift (expects 8.2.33, runner installs 8.2.34) 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
Automattic/blocks-engine#2161 ·
-
难度 1/5 1 小时以内 新手友好度 60/100
github/gh-aw-mcpg#13748 ·
-
agentic-workflows
难度 2/5 1-3 小时 新手友好度 65/100
-
难度 2/5 1-3 小时 新手友好度 70/100
personalrobotics/tsr#128 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
Azure/azure-functions-docker#1257 ·