get-helm-N installs inconsistent versions due to CDN caching
#32,329 opened on 2026/07/10
Repository metrics
- Stars
- (29,815 個のスター)
- PR merge metrics
- (平均マージ 40d 17h) (30d で 48 merged PRs)
説明
Summary
get_helm-3 queries https://get.helm.sh/helm3-latest-version to determine the latest version, but the request has no cache-control headers or cache-busting query string. CDN edge servers can serve stale cached responses for hours after a new Helm release. This means get_helm-3 running on different machines at the same time can install different versions depending on which CDN edge server they reach.
Context
We discovered this while debugging a flaky integration test in minikube: kubernetes/minikube#23323
Helm v3.21.3 was released on Jul 9, 2026 ~22:00 UTC. About 19 hours later, our CI ran get_helm.sh on multiple machines:
- Some machines installed v3.21.3 (hit updated CDN edge nodes)
- Some machines installed v3.21.2 (hit stale CDN edge nodes)
- One machine installed v3.21.2 on the first run and v3.21.3 on the second run minutes later
Impact
After a new Helm release, get_helm-3 becomes non-deterministic. Running it on different machines (or even the same machine at different times) can install different versions. This makes it an "install random version from {current-1, current}" tool rather than a reliable "install latest" tool.
This affects:
- CI/CD pipelines that run
get_helm.shon multiple nodes and expect consistent versions - Multi-machine deployments where all machines should have the same Helm version
- Any automation that compares the installed version to an expected "latest"
Current cache behavior
The version file is served from Azure Blob Storage behind Azure Front Door CDN. The origin sets no Cache-Control header:
$ curl -sI 'https://get.helm.sh/helm3-latest-version'
HTTP/2 200
date: Fri, 10 Jul 2026 18:45:44 GMT
content-type: application/octet-stream
content-length: 8
content-md5: CsRKs3WCs5c4CaZod7y6XQ==
last-modified: Thu, 09 Jul 2026 21:12:31 GMT
etag: 0x8DEDDFECA26742A
x-ms-request-id: f772980b-101e-000f-169c-108358000000
x-ms-version: 2009-09-19
x-ms-lease-status: unlocked
x-ms-blob-type: BlockBlob
x-azure-ref: 20260710T184544Z-16cf95fd5c9dfrdjhC1FRAss740000001uhg000000001dap
x-fd-int-roxy-purgeid: 4
x-cache-info: L2_T2
x-cache: TCP_REMOTE_HIT
accept-ranges: bytes
Without an explicit Cache-Control header, Azure Front Door applies its own default TTL, which can be hours. The x-cache: TCP_REMOTE_HIT and x-cache-info: L2_T2 headers confirm the response is served from a multi-tier CDN cache, not the origin.
Setting Cache-Control: max-age=60 on the helm3-latest-version blob in Azure Storage would ensure all CDN edge nodes refresh within 60 seconds of a release.
Suggested fixes
CDN configuration
Configure the CDN to use a short TTL or active cache purge for the version file (helm3-latest-version). The CDN can:
- Set a short TTL (e.g. 60 seconds) on the version file
- Actively purge the version file from all edge nodes when a release is published (most CDNs have purge APIs)
- Return
Cache-Control: no-cacheormax-age=60on the version file response, so downstream caches also keep short TTLs
The version file is ~10 bytes and queried only during install/upgrade — a short TTL has negligible cost. The binary tarballs (which are large) should stay cached with long TTLs — only the version file needs fast propagation.
Client-side cache-busting in get-helm-3
Add cache-busting to the curl/wget request in checkDesiredVersion():
# Current:
latest_release_response=$( curl -L --silent --show-error --fail "$latest_release_url" 2>&1 || true )
# Suggested:
latest_release_response=$( curl -L --silent --show-error --fail \
-H "Cache-Control: no-cache" \
"$latest_release_url?t=$(date +%s)" 2>&1 || true )
Cache-Control: no-cacheasks well-behaved intermediaries to revalidate?t=$(date +%s)forces a cache miss on CDNs that treat each unique URL as a separate cache key
This cannot guarantee bypassing all intermediate caches (ISP proxies, corporate proxies, etc.), but it handles most cases. The same change should be applied to the wget fallback path.
Issues:
- Will cause higher load on the origin server since CDN caching is bypassed; the response is ~10 bytes, so the additional load should be negligible