e2eenhancementexempt-from-staleflaky-testgood first issueroadmaptechdebttest
Repository-Metriken
- Stars
- (11.845 Sterne)
- PR-Merge-Metriken
- (Durchschn. Merge 5T 14h) (109 gemergte PRs in 30 T)
Beschreibung
What problem are you facing?
The e2e tests for Crossplane have not been as reliable as we'd like lately. For example, bumping the crossplane-runtime dependency in preparation for the v1.16 release was delayed due to e2e test failures that had been present in master for awhile (https://github.com/crossplane/crossplane/pull/5652#issuecomment-2097794718).
Ideally, we would be getting consistent green builds where all tests pass, but recently we've been getting more failures that are more likely to be test issues as opposed to genuine issues in the code we ship.
The less reliable our tests are, the less confidence we have in what we're shipping.
How could Crossplane help solve your problem?
This is a high level epic for improving our confidence and reliability in the e2e tests.
Framework improvements
- Set up buildpulse to start tracking test results (https://github.com/crossplane/crossplane/pull/5668)
- https://github.com/crossplane/crossplane/issues/5672
- https://github.com/crossplane/crossplane/pull/6090
- https://github.com/crossplane/crossplane/pull/6405
- https://github.com/crossplane/crossplane/pull/5722
- https://github.com/crossplane/crossplane/issues/4257
- https://github.com/crossplane/crossplane/pull/6160
- Bump
sigs.k8s.io/e2e-frameworkdependency tov0.5.0(https://github.com/crossplane/crossplane/pull/5722#issuecomment-2515948895)
Flaky tests
- https://github.com/crossplane/crossplane/issues/6413
- https://github.com/crossplane/crossplane/issues/6451
- more flaky tests from https://app.buildpulse.io/@crossplane/crossplane
- https://github.com/crossplane/crossplane/issues/6245
- https://github.com/crossplane/crossplane/issues/6085
- https://github.com/crossplane/crossplane/pull/6182
- https://github.com/crossplane/crossplane/issues/5228
-
TestCompositionInvalidComposedhttps://github.com/crossplane/crossplane/issues/5671#issuecomment-2105432936 -
TestConfigurationWithDependencyhttps://github.com/crossplane/crossplane/pull/5652#issuecomment-2097794718