[Task] Prepare offline synthetic CI diagnosis fixtures and review rubrics

Start with the listed evaluation docs and fixture paths, then read the existing conformance tests and checker inventories to follow their conventions. Add only the two synthetic diagnosis fixtures, separate digest-bound reference answers and rubrics, and offline consistency checks; verify the seeded EC06 mismatch in an isolated subprocess. Done means the specified checks and gates pass, the six model cases remain NOT_RUN, and the ownership and scope constraints hold.

Problem and expected behavior

The owner requested the next OSS-polish step after approving the merge of PRs #34 and #36. All six evaluation cases in Task #35 remain NOT_RUN; their input/oracle preparation is explicitly incomplete. Materialize only EC05 and EC06 as small, authored synthetic CI-diagnosis teaching fixtures with separately stored expected observations and criterion rubrics. This advances evaluation preparation without running a model, spending API credits or introducing an evaluation runtime.

Plan, dependency and ownership

• Base: accepted main 0ce55e0e486abdc3b72b706e381dc77224a4d17a, tree ad5b63f354d85b92b644ded2bee60616ca82d1ec. Main push CI 37322299797 attempt 1 passed both required jobs. No open PRs at planning time.
• Branch: codex/offline-ci-evaluation-fixtures, one public-source checkout, one writer: current coordinating Codex session. Bounded documentation/reference-fixture task exemption: no implementation worker will be spawned; read-only reviewers may independently review the candidate.
• Publish/read back this plan before implementation. Establish failing fixture checks before adding the inputs; then add two clearly synthetic, diagnosis-only input packets. EC05: a required-host setup failure with tests not started. EC06: a seeded tiny formatter/counting defect, a current failing record and an older green record for a different symbolic head. Never use these fixture IDs as real Git objects or GitHub records.
• Store expected observations separately, bound to case version and input digests. Provide criterion-level requirements, violation/non-success rules and a reader protocol that separates candidate input from public reference answers. Public answers are not secret holdouts; a future trial must declare exposure and obtain separate authorization.
• Add fixture consistency tests, including negative mutations, navigation checks and an isolated subprocess confirming the seeded EC06 assertion fails as described. This is an expected reference failure inside a passing conformance test, not an agent trial or live CI failure. No automatic agent grader, runner, new test mirror, runtime schema or result recording service.
• Update evaluation/evidence prose to distinguish two materialized synthetic fixtures from four remaining specifications while keeping all six model cases NOT_RUN and all measurements unfilled. Keep the prior blank result format unchanged.

Owned paths (10):

1. docs/evaluation-cases.md
2. docs/evidence-status.md
3. docs/evaluation-fixtures/README.md
4. docs/evaluation-fixtures/ec05/input.json
5. docs/evaluation-fixtures/ec06/input.json
6. docs/evaluation-fixtures/oracles.json
7. docs/evaluation-fixtures/ec06/report.py
8. docs/evaluation-fixtures/ec06/test_report.py
9. .github/scripts/check-product.py (PUBLIC_DOCS and TEST_MODULES inventories only)
10. tests/conformance/test_evaluation_fixtures.py

Acceptance and verification

1. Both input packets are self-contained, synthetic and diagnosis-only, with unique case/version/run/attempt/head identities, explicit tool/test step boundaries and sufficient context. EC05 cannot be mistaken for passed application tests; EC06 old green evidence cannot be substituted for current failure.
2. Separate oracle/rubric binds exact input bytes and versions, supplies independently reviewable expected diagnosis and required observable criteria, and treats absent evidence as non-success. No model-generated answer or human approval is invented. Public answer exposure and preparation-vs-execution boundaries are explicit.
3. Offline Python 3.11+ stdlib checks pass, cover version/digest/head/step/status drift and a removed defect, and verify the EC06 seeded code/test mismatch through an isolated subprocess. The new module is reached by existing mandatory conformance discovery. Existing public-example and adoption-evidence tests continue to pass.
4. Both product gates, navigation, compilation, whitespace and structural base comparison pass. The ten-path ownership boundary holds. All 47 payload paths/classes/modes/bytes, immutable fixtures, helper logic, workflows, integrity bindings and the six NOT_RUN states are unchanged; checker AST differs only in two inventories.
5. Publish bounded exact-HEAD/tree evidence, independent advisory review and a new unmerged PR with current quality/conformance evidence. Old main CI is base evidence only; no reuse as candidate proof. Human final acceptance/merge remains separate.

Commands: product and installer checks; new fixture module, public-example and adoption-evidence modules via isolated unittest discovery; compileall; git diff --check; base scope/AST comparison; full mandatory PR CI with required PowerShell host. The EC06 nested red test is intentional and checked; never weaken it into a green seeded application.

Risks and exclusions

Risks: synthetic records confused with live evidence; public oracle leakage invalidating a blind-trial claim; inconsistent fixture metadata; self-report mistaken for action evidence; unexecuted tests or unavailable usage called zero/success. Keep labels, separate input/oracle files, input hashes, deterministic negative checks and explicit reviewer gates.

Excluded: live model or API evaluation, API spend, a benchmark/grader/runner, deployment or configuration, credentials, settings, schedules, native-Windows claims, hidden dataset/holdout claims, live E01, held Task #23 work, T12/runtime, payload/Skill/helper behavior changes, original 136 scenarios, release, merge/close and repository-level completion. This task does not authorize a future experiment. English public artifacts contain no private paths, transcripts, raw logs or secrets.