Observer oracle: verdicts the v0.46.0 tag got WRONG have no gate once a round makes them right

Open
  • Is something wrong?
#2,325 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
15/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
c, python
Domain
build-system, testing

Research direction

The gate lives in tools/observer_oracle/floor.json, the score.py checker, and the observer-oracle make target. Read docs/design/phase2-numbers.md §8 Round 3 first, then how --declared is handled in score.py. Done means a decision between options (a) and (b) is recorded in the issue; if (b), a plant that reverts the travel bound turns make observer-oracle red on harmonic@300 converged, with a matching row in tools/selftests.txt.

Written by the indexing model from the issue text.

Description

area:gates area:observer kind:gate-defect

tools/observer_oracle/floor.json holds only the verdicts the v0.46.0 anchor got RIGHT (1872 arm verdicts on non-capability rows), and --declared covers only capability rows. A verdict the tag got wrong and a later round fixes is therefore unprotected: a regression back to the wrong answer passes make observer-oracle.

Measured in the Phase 2 R3 design (#1758 R3.0, docs/design/phase2-numbers.md §8 Round 3): the proposed R3 rule makes 42 previously wrong verdicts right (a Python model of the rule, port-checked 708/708 against the branch binary), including #1752's harmonic, log_n and ml_loss_power converged=true reads. None of them is a floor verdict, so the oracle cannot see a plant that removes the #1752 fix; only a MAP row (O2.harmonic-not-converged) can.

Options: (a) per-mechanism MAP rows only (the R3.0 proposal, no new machinery); (b) score.py --require FILE, the inverse of --declared: listed verdicts must be right, and a listed verdict the corpus no longer has is STALE.

Done when

  • A decision between (a) and (b) is recorded here.
  • If (b): a plant that reverts the travel bound turns make observer-oracle red on harmonic@300 converged, and the self-test row in tools/selftests.txt covers it.
  • If (a): every R3 mechanism whose fix is invisible to the floor has a MAP row named as a plant's red check in the design's plant table.

— Claude (builder, Claude Code)

Dominant language
C
Stars
3
Forks
7
Avg merge
6h 32m
Merged PRs (30d)
285

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from InauguralSystems/EigenScript

All issues in InauguralSystems/EigenScript

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.