/shutdown/ready can fail to return an HTTP response when readiness takes longer than the server write timeout
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Read internal/cmd/envoy/shutdown_manager.go, especially the HTTP server and /shutdown/ready handler around lines 43–140; then review the readiness timeout generation in internal/infrastructure/kubernetes/proxy/resource.go around lines 280–290. Reproduce the empty response with a long --ready-timeout and a delayed marker, then verify the endpoint returns HTTP 200 when the marker appears and an HTTP error when readiness times out.
Written by the indexing model from the issue text.
Description
Description:
The shutdown manager’s HTTP server configures WriteTimeout: 10s, while its /shutdown/ready handler can wait substantially longer for /tmp/shutdown-ready.
For example, configuring spec.shutdown.drainTimeout: 165s generates --ready-timeout=175s. If the marker appears after the HTTP write deadline, the handler detects readiness but the client can receive an empty response instead of HTTP 200.
Expected behavior: the endpoint returns HTTP 200 when the marker appears within readyTimeout, or its HTTP error response when that timeout expires.
Relevant implementation:
Suggested fix:
Extend the write deadline specifically for /shutdown/ready using http.NewResponseController(w).SetWriteDeadline(...) before polling begins. Set it to readyTimeout plus sufficient allowance for the final polling interval and response transmission, for example 5 seconds. Other endpoints can retain the existing server write timeout.
Previous closed, unmerged PRs address the same problem:
- #10186: endpoint-specific write deadline and real HTTP regression tests.
- #10074: server-wide write timeout adjustment.
Repro steps:
-
Start the shutdown manager from the unmodified
envoyproxy/gateway:v1.10.0image with an empty, writable/tmp:envoy-gateway envoy shutdown-manager --ready-timeout=175s -
Once
/healthzresponds, send a request from a client that can reach the manager:curl --http1.1 --max-time 40 -v \ http://<manager>:19002/shutdown/ready -
Approximately 20 seconds after the request starts, create the marker in the manager’s filesystem:
touch /tmp/shutdown-ready -
Observe the client response and manager logs.
Use a fresh manager process and initially absent marker for each run.
Observed results:
| Marker created after | Request duration | Result |
|---|---|---|
| Approximately 2 seconds | 2.06 seconds | HTTP 200 |
| Approximately 20 seconds | 20.11 seconds | curl exit code 52; no HTTP response |
Environment:
- Envoy Gateway: v1.10.0
- Image:
envoyproxy/gateway:v1.10.0 - Container platform: Linux AMD64, running through Apple Container on macOS
- Shutdown manager:
--ready-timeout=175s - Client protocol: HTTP/1.1
- Envoy proxy: not started; this reproduction exercises the shutdown manager independently
- Kubernetes: not required for this reproduction
Logs:
Client output for the approximately 20-second case:
curl: (52) Empty reply from server
Corresponding shutdown-manager log messages:
06:43:08.439 received shutdown ready request
06:43:28.510 shutdown readiness detected
No Envoy access logs are produced because the request goes directly to the shutdown manager.
- Dominant language
- Go
- Stars
- 3.1k
- Forks
- 940
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 177
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from envoyproxy/gateway
-
OIDC pass-through matcher incorrectly treats the Bearer scheme as case-sensitivePossibly taken @floriankoch claimed this 44 days ago. Opentriage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
envoyproxy/gateway#10104 ·
Maintainers usually reply within 1 day
-
provider runner swallows the manager error, leaving the controller alive without a health endpoint after losing the leaseMay be free again A pull request for this issue was closed without being merged. Opentriage
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
envoyproxy/gateway#10064 · 1 comment ·
Maintainers usually reply within 1 day
-
HTTP route rule processing does not check for DirectResponse or Redirect when filter processing has an errorPossibly taken @vshalpnjabi claimed this 9 days ago. Openhelp wanted kind/bug triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
envoyproxy/gateway#9454 ·
Maintainers usually reply within 1 day
-
is isHTTPRouteFilterReferencingSecret too eager ?May be free again A pull request for this issue was closed without being merged. Openstale triage
Difficulty 1/5 1-3 hours Newbie friendliness 72/100
envoyproxy/gateway#9339 · 1 comment ·
Maintainers usually reply within 1 day
-
stale triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
envoyproxy/gateway#8996 · 2 comments ·
Maintainers usually reply within 1 day
All issues in envoyproxy/gateway
Similar issues
-
book-layout.spec fails when the similar-books rail holds other projects' booksPossibly taken @praetorianer777 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
praetorianer777/GoTome#373 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
-
Adding an LDAP group reports every search failure as "LDAP Group DN is not found"Possibly taken @Daksha1611 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100