[helm] Publish not-ready addresses on the tablet headless Service so restarting servers stay resolvable
维护者通常 2 天内回复
评估
- 难度
- 2/5
- 预计耗时
- 1-3 小时
- 新手友好度
- 76/100
- Issue 类型
- 功能
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
- 技术栈
- helm, java, kubernetes
- 领域
- devops, infrastructure
调研方向
阅读 templates/svc-tablet.yaml，了解 chart 如何公开服务设置；检查 coordinator 的 headless Service 模板和 values，找到对应的配置。为两个服务添加所请求的默认启用设置，然后渲染 chart，确认两个 headless Service 会发布尚未就绪的地址，而 metrics Service 保持不变。
由索引模型根据 Issue 内容生成。
描述
Search before asking
- I searched in the issues and found nothing similar.
Description
Search before asking
- I searched in the issues and found nothing similar.
Motivation
Clients reach tablet servers through the CLIENT listener the chart advertises: <pod>.tablet-server-hs.<namespace>.svc.cluster.local. tablet-server-hs is a headless Service without publishNotReadyAddresses, so Kubernetes publishes a pod's DNS record only while the pod is Ready.
The chart's readiness probe is a cluster-health gate: a restarted tablet server stays NotReady until the whole cluster is GREEN (all replicas in sync, all leaders active), up to healthCheckTimeoutSeconds (1200 s by default). During that window the server is running, is registered in ZooKeeper, takes part in replication and can be named as a leader in metadata, but its hostname doesn't resolve.
What we saw on 1.0.0 during a rolling restart (3 tablet servers, ~11 minutes NotReady per server):
- The lake tiering job failed over twice with
java.net.UnknownHostException: tablet-server-2.tablet-server-hs.<ns>.svc.cluster.local. - Writers that still routed requests to the restarting server failed with
UnknownHostExceptionfor the whole window, and their metadata refreshes against that server failed the same way.
Setting publishNotReadyAddresses: true on the Service by hand made the restarting server resolvable immediately and removed the UnknownHostException errors. This is the usual pattern for StatefulSet peer-discovery Services, which need per-pod DNS regardless of readiness. The deprecated service.alpha.kubernetes.io/tolerate-unready-endpoints annotation is not an alternative: it's ignored by the EndpointSlice controller (tested on Kubernetes 1.35).
It also becomes a requirement if the INTERNAL listener is advertised by DNS name instead of pod IP (#2076): otherwise peers can't reach a restarting server during the gate either.
Readiness still controls rollouts: the StatefulSet controller and PodDisruptionBudgets use pod readiness, not the Service's endpoints, so the gate keeps working.
Solution
- Add
publishNotReadyAddresses: truetotemplates/svc-tablet.yaml, configurable throughtablet.service.publishNotReadyAddresses(defaulttrue). - Do the same for
coordinator-server-hs(coordinator.service.publishNotReadyAddresses), which is the clients' bootstrap address. - Leave the metrics Services unchanged.
Willingness to contribute
- I'm willing to submit a PR!
- 主要语言
- Java
- 星标
- 2.2k
- 派生
- 658
- 平均合并
- 2 天 22 小时
- 30 天内合并 PR
- 117
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
apache/fluss 的其他 Issue
-
[lake/iceberg] Reserved write-mode properties can be set and reset through their `iceberg.` key可能已有人在做 @aphenix-r7 于 15 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 2 天内回复
-
[common] Decimal#hashCode is inconsistent with Decimal#equals可能已有人在做 @firasbouzazi 于 27 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 2 天内回复
-
[security] PlainSaslServerProvider conflicts with Hadoop SASL provider causing authentication failure可能已有人在做 @xinzhang-hello 于 33 天前认领。 未关闭
难度 1/5 1 小时以内 新手友好度 88/100
维护者通常 2 天内回复
-
[server] KV snapshot scheduling stops permanently after a transient snapshot-init failure可能已有人在做 @zhang-arvin 于 32 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 2 天内回复
-
[docs] Document advertised-resource and ACL principal ignore-case options可能已有人在做 @EchoLee5 于 45 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 2 天内回复
相似的 Issue
-
status: waiting-for-triage
难度 2/5 1-3 小时 新手友好度 78/100
spring-projects/spring-framework#37614 ·
维护者通常 1 天内回复
-
azure-spring azure-spring-aad Client
难度 2/5 1-3 小时 新手友好度 72/100
Azure/azure-sdk-for-java#51046 · 1 个 reaction ·
维护者通常 1 天内回复
-
helm-knative-apply.sh fails with Helm 4可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭area: triage theme: undefined
难度 2/5 1-3 小时 新手友好度 85/100
jhipster/generator-jhipster#35895 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 85/100
oracle/graalvm-reachability-metadata#11659 ·
维护者通常 1 天内回复