[PERF] Noticeable perf regression over time

As part of the ASP.NET team, I am measuring envoy and the perf of other proxies continuously. For the past few months we can see that envoy's perf is slowly degrading. Overall, compared to nginx or haproxy it's still doing very well, but you might have missed these numbers so I am sharing them with you.

The following chart represents RPS and mean latency for different payload sizes overtime (10B, 100B, 1KB).

This is based on the envoyproxy/envoy-dev:latest docker image, and run around once a day using this tag. The full configuration can be found here: https://github.com/aspnet/Benchmarks/tree/master/docker/envoy The benchmark is using bombardier with 256 connections, using 3 different machines (load, proxy, downstream), with dedicated 40Gb/s NICs on a private lan and a dedicated switch. Other proxies like nginx, haproxy, or even the baseline (without proxies) are stable during this time frame.