envoyproxy/envoy

EDS multiplexing causes EDS initial fetch failure

Open

#25,195 opened on Jan 26, 2023

View on GitHub
 (5 comments) (0 reactions) (1 assignee)C++ (5,373 forks)batch import
area/edsbughelp wanted

Repository metrics

Stars
 (27,997 stars)
PR merge metrics
 (Avg merge 8d) (378 merged PRs in 30d)

Description

Description:

We're running a GRPC SoTW control plane for xDS. When we deployed a changeset which included https://github.com/envoyproxy/envoy/pull/22419, Envoy failed to fetch EDS from the control plane at all, timing out on initial EDS fetch. When we disabled the feature with the runtime flag envoy.reloadable_features.multiplex_eds and restarted Envoy, EDS worked properly again.

Apart from EDS failures, we saw many occurrences of this log message which has never come up before (looks like maybe 1 per EDS cluster):

[warning][config] [external/envoy/source/common/config/grpc_stream.h:65] gRPC bidi stream to envoymanager for rpc StreamEndpoints(stream .envoy.service.discovery.v3.DiscoveryRequest) returns (stream .envoy.service.discovery.v3.DiscoveryResponse); already exists!

The PR was reverted in https://github.com/envoyproxy/envoy/pull/25157 for a (maybe unrelated) reason so I'm filing this just for SA when the EDS muxing change goes back in.

Contributor guide

Research direction
Examine the EDS multiplexing implementation in the reverted PR and trace the gRPC stream warning to identify the cause of initial fetch failure.
Tech stack
cppgrpc
Domain
backend
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GitC++gRPC
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20