The dev-box cost column (max_local) is enforced by no job and had gone out of date: 6 rows over and 26 stale on phase2-numbers

Aperta
#2,443 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Refactoring
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
c, shell

Direzione di ricerca

Start with docs/design/phase2-numbers.md, the header and behavior of tools/phase2_cost.sh, and the hosted bench workflow that enforces max_ci. Decide and document whether a hosted lane should enforce max_local or the column should be removed and local runs made report-only. If retaining it, measure the three named rows on the branch head, explain the changes behind the two unobserved_armed_loop costs, and ensure the local gate exits 0; also address when --n6-text is required.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

needs-triage

What. tests/phase2/cost_expect.txt has two columns of declared maxima, each "from its own gate": max_ci, which the hosted bench job enforces on every run, and max_local, which only tools/phase2_cost.sh <base> <cand> enforces. No workflow runs the second. On the branch phase2-numbers it had gone out of date in both directions, and nothing could have shown it.

Measured on the development machine, main at 96e15c33 as the base, tools/phase2_cost.sh --n6-text, 32 rows:

the branch at OVER its max_local STALE (more than 0.5 under it)
6645d9cf (before #2436) 6 26
#2436's head cd850ae4 (its 28 stale rows rewritten from this run) 3 0

The three that remain OVER, which #2436 leaves as they are because its bar forbids raising a maximum:

row delta max_local before #2436
bench/unobserved_armed_loop.eigs jit +90.8566% 67.7 +90.8569%
bench/unobserved_armed_loop.eigs off +23.5725% 9.8 +23.5727%
bench/observer_gate_soft_keyword.eigs jit +0.9554% 0.3 +38.9988%

The first two are not moved by #2436 (their hosted readings are unchanged too). The third fell from +39.00% to +0.96% with #2436 and is still above a maximum that was already false before it. A hosted container gives the same deltas as the development machine to four digits (+90.84% and +23.57% for the first two), so the column is not a property of one machine: it is one method's numbers that nobody was running.

Why it matters. A two-sided gate exists so that a recovered cost brings its declaration down and a new regression cannot hide under a loose bound. A column that no job runs gives neither: at 6645d9cf the local gate fails on all 32 rows (6 over, 26 stale), so nobody who ran it could have seen a regression of their own in it.

Done when

  • One of the two, decided and written in docs/design/phase2-numbers.md and the header of tools/phase2_cost.sh: (a) a hosted lane runs tools/phase2_cost.sh with its own column on the branch, so the column is kept by a machine; or (b) the max_local column goes, the tool reads the one hosted column, and the local run becomes a report with no verdict of its own.
  • If the column stays: the three rows above are measured on the branch's head and declared by the rule with the reason for each (what moved unobserved_armed_loop from 67.7 to 90.9 and from 9.8 to 23.6, by which change), and the local gate exits 0 on that head.
  • The cost comparison needs --n6-text when the base is main (one hosted builder's run stopped at tests/bench_idxset.eigs without it): the tool's header says so, or the tool chooses it when the two binaries are of different number models.

Refs #2433, #2436, #1939.

— Claude DevBox

Lingua principale
C
Stelle
3
Fork
7
Merge medio
3h 28m
PR unite (30g)
270

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di InauguralSystems/EigenScript

Tutte le issue di InauguralSystems/EigenScript

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.