CI: skip unaffected C++ test group on PRs and speed up slow C++ tests

Aperta
  • Qualcosa non va?
#2,258 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
32/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
cpp, github-actions, yaml
Ambito
ci-cd, testing-qa

Direzione di ricerca

Start with the changed-files job and the conda-cpp-tests-* jobs in pr.yaml, then the pr-builder and pr-test-summary jobs that must accept a skipped job. For the speed-up, the timings name C_API_TEST and DeterministicBBTestFixture.deterministic_reproducibility/1, plus NUMOPT_INTERNAL_TEST and ROUTING_INTERNAL_TEST. Done means the path-filtered skip works for both sides, shared code still runs both groups, and the heavy tests run faster on v100. The open questions on which tests belong on PRs need maintainer input first.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Follow-up to #2247 (split C++ tests into routing and mathopt jobs), raised by @bdice and @KyleFromNVIDIA in review.

1. Skip the unaffected group based on changed files (@bdice)

Run only the routing or only the mathopt C++ test job when a PR touches just that side.

  • Add routing and mathopt path filters to the changed-files job in pr.yaml, and gate each conda-cpp-tests-* job with them.
  • Shared code (utilities, C API, common cpp/src and cpp/include) must still run both groups.
  • pr-builder / pr-test-summary must tolerate a skipped job.

2. Make the tests faster before relying on sharding (@KyleFromNVIDIA)

Per-binary timings from PR #2247 CI run 41485888506 (commit 5722b9a4). Four binaries make up about 90% of test time in each group.

Binary Group h100 v100
NUMOPT_INTERNAL_TEST mathopt 542s 1122s
INCUMBENT_CALLBACK_TEST mathopt 152s 202s
GRPC_INTEGRATION_TEST mathopt 102s 117s
C_API_TEST mathopt 64s 232s
ROUTING_INTERNAL_TEST routing 659s 717s
ROUTING_UNIT_TEST routing 155s 155s

Candidates to investigate:

  • C_API_TEST on v100: DeterministicBBTestFixture.deterministic_reproducibility/1 takes 181s on v100 vs about 25s on h100, and is why v100 mathopt takes 36 min against 16-19 min elsewhere. Use a smaller instance or a lower work limit.
  • Duplicate coverage: deterministic B&B reproducibility runs in both NUMOPT_INTERNAL_TEST (about 100s) and C_API_TEST.
  • Routing time limits: OBJECTIVE_FUNCTION/0 and /1 take about 75s each, and the 14 CVRPTW_Retail cases about 20s each. Check whether a lower search time limit keeps results stable.
  • Large LP/MIP sweeps: mip_solve.run_small_tests (133s) and pdlp_class.run_sub_mittleman (106s). Consider a fast subset for PRs and the full set for nightly.
  • GRPC_INTEGRATION_TEST: PathSelectionTests (42s) and ErrorRecoveryTests (24s). Check for fixed sleeps or restarts.
  • Per-job setup: about 4.5 min of setup and dataset download per job, paid twice after the split.

Open questions

  • Which of the heavy tests are required for PR coverage and which can move to nightly?
  • Are the v100 slowdowns from the GPU, CPU count, or time-limit behaviour?
Lingua principale
Cuda
Stelle
1.1k
Fork
249
Merge medio
1g 15h
PR unite (30g)
124

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/cuopt

Tutte le issue di NVIDIA/cuopt

Issue simili

Altre issue su DevOps

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.