Round 3: the leave-one-out table the pre-registration asks for (converged, stable, improving)
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
調査の方向性
Start by reading the Round 3 pre-registration in #1758, the frozen commit 5b93b236, and tools/observer_oracle/judge.py; the issue names src/eigenscript.c, docs/design/phase2-numbers.md, corpus.json, labels.json, and docs/PREDICATES.md. Build and score each of the five single-mechanism arms on the specified H1 data, then have a second party independently recompute the table. Done means the posted table includes the requested scores, intervals, effects, residual, and all-readings figures, with the record unchanged.
索引モデルが issue の本文から書いたものです。
説明
The pre-registration of Round 3 (#1758, comment 6430031838, "LOO ablation") asks for a leave-one-out table for every predicate whose balanced accuracy on the held-out set differs between Round 3's build and the incumbent by 0.02 or more. After the reveal those are converged (+0.034), stable (+0.131) and improving (+0.264). The table is a registered duty of the round. It is scored from Round 3's frozen commit and decides nothing about what ships (RULING R30R-2 already did), so it is kept apart from the work that brings the branch to what ships (#2407).
This text is a first draft of a bar. It gets a blind review before a builder gets it, and the reviewed text replaces this one.
What the table is
Arms, each a build: the incumbent INC (v0.47.0), Round 3's final commit R3 (5b93b236), and for each of the five mechanisms the pre-registration names, R3 with that one mechanism reverted and nothing else:
| mechanism | where it is in 5b93b236 |
|---|---|
| O1: the distance signal, entropy removed | the recorder, the readout and the snapshot's routing in src/eigenscript.c |
| the scale rule of #1580 | obs_floor_scale; the experiment adopted no rule, so the function returns the incumbent's scale |
the conservative converged of #1752 |
not built: the final converged is the standard diameter test |
| the integer path of #1830 | exact steps, the test for a window of exact integers, no noise floor for such a window, the exact branches of the two settled tests |
| O2: the definitions aligned with the standard tests | the bodies of the numeric tests and their branches for values that are not finite |
A mechanism that Round 3 did not build, or that is the incumbent's already, is a row all the same: its arm has the bytes of R3, its effect is zero, and the row says where that is recorded (docs/design/phase2-numbers.md).
Done when
- For each mechanism that exists, a patch against
5b93b236that reverts it alone, with the list of functions it touches, kept on a branch that is never merged. Each arm builds, and--floorexits 0 for it or the row says which calibration answers it loses. - Each arm is scored on H1 as the reveal's tag
phase2-r3-revealhas it (corpus.jsonsha25685be07d9…,labels.jsonsha25622ed1499…; never generated again, #2404), withtools/observer_oracle/judge.py, seed 1758 and 10,000 resamples, through a manifest of its own beside the record. The record's files do not change. - The table, posted on #1758: for
converged,stableandimproving, the balanced accuracy ofINC, ofR3and of each of the five arms on the 735 readings bothINCandR3have, each with its interval; the effect of a mechanism, BA(R3) minus BA(R3without it); and the residual, BA(R3) minus BA(INC) minus the sum of the five effects. The scores on all 738 readings are given beside it and not mixed into it. The figures ofINCandR3are the reveal's own, to the digit. - A second party recomputes the table from the arms' raw readings with a script of its own, and the two agree to the digit.
- Nothing in the table is used to choose a test:
docs/PREDICATES.mdgets the table as a record, with that sentence.
— Claude DevBox
- 主要言語
- C
- スター
- 3
- フォーク
- 7
- 平均マージ
- 3時間 28分
- マージ済み PR(30日)
- 270
環境構築
このプロジェクトの開発コンテナを、あなたの GitHub アカウントでブラウザ上に起動します。
- Dockerfile または Docker Compose ファイルあり
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
InauguralSystems/EigenScript のほかの issue
-
area:gates kind:gate-defect
難易度 2/5 1〜3時間 初心者へのやさしさ 83/100
InauguralSystems/EigenScript#2476 ·
メンテナーはふだん 1 日以内に返信
-
area:gates needs-triage
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
InauguralSystems/EigenScript#2462 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
The hosted-RNG check refuses a clean tree that has a tracked file with a double quote in its nameオープンarea:gates kind:gate-defect needs-triage
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
InauguralSystems/EigenScript#2461 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
area:gates kind:gate-defect needs-triage
難易度 2/5 1〜3時間 初心者へのやさしさ 86/100
InauguralSystems/EigenScript#2458 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
`http_post`: `headers` text with something after its JSON value now means no headers, without a wordオープンarea:http kind:silent-wrong
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
InauguralSystems/EigenScript#2394 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
InauguralSystems/EigenScript の issue をすべて見る
似ている issue
-
難易度 1/5 1時間未満 初心者へのやさしさ 92/100
flipperdevices/flipperzero-firmware#4520 ·
メンテナーはふだん 1 日以内に返信
-
area/build-framework kind/enhancement priority/low
難易度 2/5 1〜3時間 初心者へのやさしさ 77/100
yugabyte/yugabyte-db#38826 ·
メンテナーはふだん 1 日以内に返信
-
難易度 1/5 1時間未満 初心者へのやさしさ 88/100
meshcore-dev/MeshCore#3891 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
bug grammar sev-3
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
-
BUG/MEDIUM: regex: worker SIGSEGV when a regex is applied to an empty body on libc-regex buildsオープンstatus: needs-triage type: bug
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100