The dev-box cost column (max_local) is enforced by no job and had gone out of date: 6 rows over and 26 stale on phase2-numbers

オープン
#2,443 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
35/100
issue の種類
リファクタリング
明瞭さ
おおむね明確
活発さ
活発
技術スタック
c, shell

調査の方向性

Start with docs/design/phase2-numbers.md, the header and behavior of tools/phase2_cost.sh, and the hosted bench workflow that enforces max_ci. Decide and document whether a hosted lane should enforce max_local or the column should be removed and local runs made report-only. If retaining it, measure the three named rows on the branch head, explain the changes behind the two unobserved_armed_loop costs, and ensure the local gate exits 0; also address when --n6-text is required.

索引モデルが issue の本文から書いたものです。

説明

needs-triage

What. tests/phase2/cost_expect.txt has two columns of declared maxima, each "from its own gate": max_ci, which the hosted bench job enforces on every run, and max_local, which only tools/phase2_cost.sh <base> <cand> enforces. No workflow runs the second. On the branch phase2-numbers it had gone out of date in both directions, and nothing could have shown it.

Measured on the development machine, main at 96e15c33 as the base, tools/phase2_cost.sh --n6-text, 32 rows:

the branch at OVER its max_local STALE (more than 0.5 under it)
6645d9cf (before #2436) 6 26
#2436's head cd850ae4 (its 28 stale rows rewritten from this run) 3 0

The three that remain OVER, which #2436 leaves as they are because its bar forbids raising a maximum:

row delta max_local before #2436
bench/unobserved_armed_loop.eigs jit +90.8566% 67.7 +90.8569%
bench/unobserved_armed_loop.eigs off +23.5725% 9.8 +23.5727%
bench/observer_gate_soft_keyword.eigs jit +0.9554% 0.3 +38.9988%

The first two are not moved by #2436 (their hosted readings are unchanged too). The third fell from +39.00% to +0.96% with #2436 and is still above a maximum that was already false before it. A hosted container gives the same deltas as the development machine to four digits (+90.84% and +23.57% for the first two), so the column is not a property of one machine: it is one method's numbers that nobody was running.

Why it matters. A two-sided gate exists so that a recovered cost brings its declaration down and a new regression cannot hide under a loose bound. A column that no job runs gives neither: at 6645d9cf the local gate fails on all 32 rows (6 over, 26 stale), so nobody who ran it could have seen a regression of their own in it.

Done when

  • One of the two, decided and written in docs/design/phase2-numbers.md and the header of tools/phase2_cost.sh: (a) a hosted lane runs tools/phase2_cost.sh with its own column on the branch, so the column is kept by a machine; or (b) the max_local column goes, the tool reads the one hosted column, and the local run becomes a report with no verdict of its own.
  • If the column stays: the three rows above are measured on the branch's head and declared by the rule with the reason for each (what moved unobserved_armed_loop from 67.7 to 90.9 and from 9.8 to 23.6, by which change), and the local gate exits 0 on that head.
  • The cost comparison needs --n6-text when the base is main (one hosted builder's run stopped at tests/bench_idxset.eigs without it): the tool's header says so, or the tool chooses it when the two binaries are of different number models.

Refs #2433, #2436, #1939.

— Claude DevBox

主要言語
C
スター
3
フォーク
7
平均マージ
3時間 28分
マージ済み PR(30日)
270

環境構築

Codespaces で開く

このプロジェクトの開発コンテナを、あなたの GitHub アカウントでブラウザ上に起動します。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

InauguralSystems/EigenScript のほかの issue

InauguralSystems/EigenScript の issue をすべて見る

似ている issue

C の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。