help wantedkind/flakelifecycle/frozen
Repository metrics
- Stars
- (9,013 stars)
- PR merge metrics
- (PR metrics pending)
Description
An analysis of test flakes showed that examples tests are some of the flakiest tests in this repo. Many of them are redundant, use more resources than necessary, or don't actually verify the results being tested. Removing redundant tests, reducing resources requested, and rewriting integration tests as unit tests where possible will cut down on resource usage of examples tests and make them faster and hopefully time out less.
Some ideas:
- Since we have conversion tests, it's probably not necessary to run all of our examples in both v1 and v1beta1.
- It's likely unnecessary to run the same tests with enable-api-fields set to "alpha", "beta", and "stable" (especially for stable features).
- Many tests have more tasks than they need (aka more pods) and use PVCs with more storage than necessary. This adds scheduling time to tests.
- Tests that use results and params typically test that certain syntax is possible but not that variable replacement has the correct value at the end.
Assigning myself to identify specific examples tests that can be eliminated.