`rate_limited_retry_back_off` does not sufficiently respect `timeout`
#24.316 aperta il 2 dic 2022
Metriche repository
- Star
- (27.997 star)
- Metriche merge PR
- (Merge medio 8g) (378 PR mergiate in 30 g)
Descrizione
Title: rate_limited_retry_back_off does not sufficiently respect timeout
Description:
I have set up rate_limited_retry_back_off with the header retry-after; this works as expected.
However, I have cases where the value retry-after is larger than the remaining timeout budget, or the value of timeout even. This is the case when we have a server side timeout due to an expensive operation, and do not want any immediate retries.
In these cases, Envoy will try to wait for the amount of time specified by the retry-after header, but will eventually time out and respond with a 504.
Since the wait time specified by retry-after is larger than the remaining time budget for the request, there is no way the request can succeed; timeout is a certainty. I believe the correct behavior would be to instantly fail the request, either with the upstream response or with a 504.
Repro steps:
You'll need a service that can generate e.g. a 503 along with a Retry-After header with a value higher than 10, ensuring the timeout is enforced.
Config:
timeout: 10s
retry_policy:
rate_limited_retry_back_off:
max_interval: 300s
reset_headers:
- format: SECONDS
name: Retry-After