envoyproxy/envoy

Memory utilization can grow significantly when struct fields are used for configuration of Envoy components

Open

#5,965 opened on Feb 15, 2019

View on GitHub
 (2 comments) (0 reactions) (0 assignees)C++ (5,373 forks)batch import
help wantedtech debt

Repository metrics

Stars
 (27,997 stars)
PR merge metrics
 (Avg merge 8d) (378 merged PRs in 30d)

Description

Title: Memory utilization can grow significantly when struct fields are used for configuration of Envoy components.

Description: In my test bootstrap config with http_connection_manager configuration in "config" field with 50K virtual hosts uses ~1807MB. The same configuration, but put in "typed_config" results in 292MB. This is due to maps being created to represent struct fields during reflection in MessageUtil::checkForDeprecation call.

[optional Relevant Links:] Test that uses "typed_config" for http_connection_manager configuration: https://github.com/dmitri-d/envoy/blob/b6b2a9cff293a1bd97166e742091900fa36ce388/test/integration/rds_memory_stats_test.cc Similar to the test above, but uses "config", which is a struct field: https://github.com/dmitri-d/envoy/blob/rds-memory-stats/test/integration/rds_memory_stats_test.cc

Contributor guide

Research direction
Investigate how struct fields trigger map creation in MessageUtil::checkForDeprecation and find a way to avoid or cache these maps to reduce memory usage.
Tech stack
cpp
Domain
backendinfrastructureperformance
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
C++Envoy internals
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20