envoyproxy/envoy

admin: making a streaming version of /clusters

Open

#32,055 opened on Jan 25, 2024

View on GitHub
 (10 comments) (1 reaction) (1 assignee)C++ (5,373 forks)batch import
area/adminarea/perfenhancementhelp wanted

Repository metrics

Stars
 (27,997 stars)
PR merge metrics
 (Avg merge 8d) (378 merged PRs in 30d)

Description

/clusters can produce a remarkably huge response. This will need to be fully buffered as an in-memory continuous string. For large configs, this will be slow and may exert memory pressure as well.

See also #31755 as well as prior work streaming admin stats in https://github.com/envoyproxy/envoy/pull/19693. This work is not trivial and has risks. See also the difficult attempt to make /stats/prometheus stream as well: https://github.com/envoyproxy/envoy/pull/24998 which needed to be rolled back due to complexities in the Prom format that made streaming difficult.

See also #32054 which is a similar request for /config_dump

Contributor guide

Research direction
Examine the current /clusters endpoint implementation and understand how it constructs the full response. Review prior streaming work in PR #19693 and note the difficulties encountered in PR #24998. Identify the data structures and serialization patterns that need to be changed to support streaming.
Tech stack
cpp
Domain
backendinfrastructurenetworking
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
C++Envoy internals
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20