envoyproxy/envoy

Set up proper caching for mobile CI

Open

#26.970 aperta il 26 apr 2023

Vedi su GitHub
 (6 commenti) (0 reazioni) (0 assegnatari)C++ (5373 fork)batch import
area/ciarea/test flakesenhancementhelp wantedmobileno stalebot

Metriche repository

Star
 (27.997 star)
Metriche merge PR
 (Merge medio 8g) (378 PR mergiate in 30 g)

Descrizione

Recently, we have added a bunch of caching to Envoy's CI which has both sped things up and fixed (or seriously mittigated) a load of persistent flakes that have been endemic (optimistically i would say until ~now)

Essentially there are a few things worth caching:

  • docker build image
  • bazel directories (repository_cache, external, .cache)

Regarding the docker image it is huge - to make it work faster than pulling from dockerhub in Envoy's CI i have had to load it using tmpfs - any other way would be significantly slower

I held off doing this for a long time because dockerhub can be faster - but now that we have done this it has been a huge improvement to both performance and reliability. Anecdotally i would say that the azp cache gets faster the more you use it whereas the reverse seems to be true with the dockerhub cache (with some caveats)

Likewise with the bazel directories, a huge number of flakes occur just because of transient network issues, and caching these mostly eliminates this problem

While working on this for Envoy one thing i noticed was that incorrect use of bazel was creating multiple environments, fixing that was a small but sure optimization. Envoy now dumps pretty good fs info at the end of the build, and this is invaluable to understanding what bazel is doing and why it goes wrong - i strongly suggest we do the same for the mobile ci

Guida contributor