The dev-box cost column (max_local) is enforced by no job and had gone out of date: 6 rows over and 26 stale on phase2-numbers

Offen
#2,443 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
35/100
Issue-Typ
Refactoring
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
c, shell
Bereich
performance, testing

Rechercherichtung

Start with docs/design/phase2-numbers.md, the header and behavior of tools/phase2_cost.sh, and the hosted bench workflow that enforces max_ci. Decide and document whether a hosted lane should enforce max_local or the column should be removed and local runs made report-only. If retaining it, measure the three named rows on the branch head, explain the changes behind the two unobserved_armed_loop costs, and ensure the local gate exits 0; also address when --n6-text is required.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

needs-triage

What. tests/phase2/cost_expect.txt has two columns of declared maxima, each "from its own gate": max_ci, which the hosted bench job enforces on every run, and max_local, which only tools/phase2_cost.sh <base> <cand> enforces. No workflow runs the second. On the branch phase2-numbers it had gone out of date in both directions, and nothing could have shown it.

Measured on the development machine, main at 96e15c33 as the base, tools/phase2_cost.sh --n6-text, 32 rows:

the branch at OVER its max_local STALE (more than 0.5 under it)
6645d9cf (before #2436) 6 26
#2436's head cd850ae4 (its 28 stale rows rewritten from this run) 3 0

The three that remain OVER, which #2436 leaves as they are because its bar forbids raising a maximum:

row delta max_local before #2436
bench/unobserved_armed_loop.eigs jit +90.8566% 67.7 +90.8569%
bench/unobserved_armed_loop.eigs off +23.5725% 9.8 +23.5727%
bench/observer_gate_soft_keyword.eigs jit +0.9554% 0.3 +38.9988%

The first two are not moved by #2436 (their hosted readings are unchanged too). The third fell from +39.00% to +0.96% with #2436 and is still above a maximum that was already false before it. A hosted container gives the same deltas as the development machine to four digits (+90.84% and +23.57% for the first two), so the column is not a property of one machine: it is one method's numbers that nobody was running.

Why it matters. A two-sided gate exists so that a recovered cost brings its declaration down and a new regression cannot hide under a loose bound. A column that no job runs gives neither: at 6645d9cf the local gate fails on all 32 rows (6 over, 26 stale), so nobody who ran it could have seen a regression of their own in it.

Done when

  • One of the two, decided and written in docs/design/phase2-numbers.md and the header of tools/phase2_cost.sh: (a) a hosted lane runs tools/phase2_cost.sh with its own column on the branch, so the column is kept by a machine; or (b) the max_local column goes, the tool reads the one hosted column, and the local run becomes a report with no verdict of its own.
  • If the column stays: the three rows above are measured on the branch's head and declared by the rule with the reason for each (what moved unobserved_armed_loop from 67.7 to 90.9 and from 9.8 to 23.6, by which change), and the local gate exits 0 on that head.
  • The cost comparison needs --n6-text when the base is main (one hosted builder's run stopped at tests/bench_idxset.eigs without it): the tool's header says so, or the tool chooses it when the two binaries are of different number models.

Refs #2433, #2436, #1939.

— Claude DevBox

Vorherrschende Sprache
C
Sterne
3
Forks
7
Ø Merge
3 Std. 28 Min.
Gemergte PRs (30 T.)
270

Entwicklungsumgebung

In Codespaces öffnen

Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus InauguralSystems/EigenScript

Alle Issues in InauguralSystems/EigenScript

Ähnliche Issues

Weitere Issues zu C

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.