envoyproxy/envoy

requests failed "above" EDS level should be reflected in load reporting

Open

#13,206 opened on Sep 21, 2020

View on GitHub
 (1 comment) (0 reactions) (0 assignees)C++ (27,997 stars) (5,373 forks)batch import
area/cluster_managerbughelp wanted

Description

There are at least a couple of cases where Envoy will fail requests "above" the EDS layer:

  • When the EDS resource does not exist.
  • When the cluster fails requests due to maintenance mode.

In these cases, the failed requests don't show up in LRS load reporting at all. I think this should be changed to report these as failed requests in an UpstreamLocalityStats message that does not have the locality field populated. This representation seems compatible with how we report stats for non-EDS clusters (e.g., LOGICAL_DNS clusters).

CC @htuch

Contributor guide

Tech stack
cpp
Domain
backendobservability
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Envoy proxy basicsLoad reporting (LRS)EDS (Endpoint Discovery Service)C++ development
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
25
Research direction
Investigate how Envoy handles requests when EDS resource is missing or cluster is in maintenance mode. Look at the load reporting code in source/common/upstream/ and the API definition in api/envoy/config/endpoint/v3/load report.proto. Consider modifying the code to emit an UpstreamLocalityStats with an empty locality field for these failures. Review existing tests for load reporting to understand the pattern.