Support HTTPRoute retry (no backend retry path exists today)
Maintainers usually reply within 5 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- go, kubernetes
- Domain
- backend, networking
Research direction
Start with the existing route cache policy plumbing from routing.json through ghost.json, then read ghost/src/vhost_director.rs and internal/vcl/postamble.vcl. Decide how retry settings trigger Varnish retries, avoid the failed backend, handle backoff, and treat non-idempotent requests; check the Gateway API experimental-field requirements. Done means retry behavior is covered by ghost-side VTC tests and an end-to-end test with a failing pod.
Written by the indexing model from the issue text.
Description
Backend retry: no retry path exists today
Field: HTTPRoute.spec.rules[].retry (HTTPRouteRetry) — Extended support, experimental channel only.
Problem
There is no backend retry anywhere in the gateway. We deliberately do not generate
vcl_backend_error (see AGENTS.md: it is reserved for user VCL), builtin VCL delivers
the 503, and nothing ever calls return(retry).
Consequence: a pod that is still in the EndpointSlice but failing to accept connections
returns 503 straight to the client, with no attempt at a sibling pod in the same backend
group. Kubernetes readiness probes eventually evict it, but that window is seconds to tens
of seconds, and for its duration a fixed fraction of requests to that route simply fail —
1/N of them per retry-free attempt, where N is the pod count in the selected group.
This is the single largest availability gap in the data path right now.
Proposed scope
Implement HTTPRouteRetry:
attempts— max retries for one backend request (maps ontobereq.retries/
max_retries)codes— response status codes that trigger a retrybackoff— minimum delay between attempts (GEP-2257 duration)
Per the API docs, implementations SHOULD retry connection errors (disconnect, reset,
timeout, TCP failure) whenever a retry stanza is configured, regardless of codes.
Implementation notes
- Plumbing: retry config is per-rule, so it rides the existing path —
routing.json→ghost.json→ ghost route entry. Same shape as the cache policy
fields already attached to routes. - Triggering the retry:
return(retry)must NOT go in the preamble's
vcl_backend_response, for the same reasonreturn(pass)does not go in the preamble's
vcl_recv— it terminates VCL and user VCL never runs. Use the established deferred
pattern: ghost sets a header (e.g.X-Ghost-Retry), and a postamble
vcl_backend_responsechecks it and callsreturn(retry)after user VCL has run.
This means adding avcl_backend_responsetointernal/vcl/postamble.vcl, which
currently only carriesvcl_recv. - Avoiding the same pod: on retry, Varnish re-enters
vcl_backend_fetchand calls
the director again.select_backend_from_groups()inghost/src/vhost_director.rsis
pure weighted random, so a retry can land on the backend that just failed. Ghost should
readbereq.retriesand exclude the previously-selected backend for this fetch.
Cheapest version: stash the last-tried backend key onbereqand filter it out of the
candidate list. This is the prerequisite for varnish/gateway#92 and makes retry actually
useful rather than statistically pointless. - Backoff: Varnish has no native inter-retry delay. May need
vmod_std's sleep
equivalent or be documented as unsupported initially — worth deciding explicitly rather
than silently ignoring the field. - Idempotency: the Gateway API retry spec does not restrict retries by method.
Decide whether to retry non-idempotent requests and document the choice.
Caveats
retryis<gateway:experimental>, so it is absent fromstandard-install.yaml, which
is what the docs tell users to install. Supporting it means either documenting the
experimental channel as a prerequisite for this feature, or gating the controller on
the field's presence.- There is no
SupportHTTPRouteRetryconformance feature and no retry conformance tests
in gateway-api v1.5.0, so this needs our own test coverage (VTC for ghost-side backend
exclusion, plus an e2e with a deliberately broken pod).
- Dominant language
- Go
- Stars
- 16
- Forks
- 3
- Avg merge
- 5d 15h
- Merged PRs (30d)
- 10
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from varnish/gateway
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 5 days
-
beta documentation
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
varnish/gateway#58 · 1 comment ·
Maintainers usually reply within 5 days
-
beta documentation
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
varnish/gateway#52 · 1 comment ·
Maintainers usually reply within 5 days
-
beta enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 5 days
-
Difficulty 5/5 Over a week Newbie friendliness 12/100
varnish/gateway#123 · 1 comment ·
Maintainers usually reply within 5 days
Similar issues
-
Date tests fail when the local date differs from the UTC datePossibly taken @unidoc-alip claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 87/100
Maintainers usually reply within 1 day
-
enhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
rossoctl/context-guru#1217 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
router-for-me/CLIProxyAPI#7659 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
netsampler/goflow2#601 ·
Maintainers usually reply within 1 day