envoyproxy/envoy

proposal: Add tracking latencies and flamegraphs in CI

Open

#11,266 opened on May 19, 2020

View on GitHub
 (23 comments) (0 reactions) (1 assignee)C++ (5,373 forks)batch import
area/perfhelp wanted

Repository metrics

Stars
 (27,997 stars)
PR merge metrics
 (Avg merge 8d) (378 merged PRs in 30d)

Description

Filing this issue to get a feel for interest in this

Goal:

Add a means to track and persist latency numbers and perf visualizations like flamegraphs over time in CI. This would allow us to track how we're doing over time as well as have perf information at hand when a latency regression is observed.

Description:

Nighthawk uses a lightweight python-based framework for integration testing. This framework serves as a basis for writing NH's own benchmarks.

With a small bit of modification this could be modified to:

  • make consumption very low friction in foreign code bases (like Envoy)
  • allow it to inject proxies in the between the client and test server. For example, Envoy at a certain sha
  • scavenge tests from external locations

More details, and some concrete scripts for getting an idea of what this would look like can be found here.

/cc @danzh2010 @htuch

Contributor guide

Research direction
Investigate Nighthawk's existing benchmark framework and CI pipeline to understand how to add latency tracking and flamegraph generation. Look at the provided prototype scripts and determine integration points.
Tech stack
python
Domain
backendinfrastructureci cd
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
CI/CDPython
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40