[Task] Publish checkpointed EC04 regression-and-fix evidence
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 15/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- markdown, python
- Domain
- documentation, testing
Research direction
Start from the owned paths: the new dated report under docs/evaluation-results/, the EC01/EC03/EC04 status lines in docs/evaluation-cases.md and docs/evidence-status.md, and the consistency assertions in tests/conformance/test_public_examples.py. Read Task #53 and PR #52 to see how an earlier report was published. Done means the eight owned files change as listed, the checker registers the report, and the offline checks pass.
Written by the indexing model from the issue text.
Description
Problem and expected behavior
The owner explicitly approved proceeding with a documentation PR after the Task #53 trial result was presented. Publish the bounded EC04 checkpointed regression/fix evidence so readers can find both the observed successes and unresolved coverage limits without relying on session history.
Source work order: Task #53.
Primary result: executed trial receipt.
Related catalog: Task #35.
Publication precedent: Task #51 / PR #52.
One attempt EC04-20261009-A1, two turns: C1/C2/C3 PASS; C4 and overall UNCHECKABLE. Publication must not upgrade that result, rewrite historical reports, or confuse source-specification NOT_RUN fields with the separately executed attempt.
Scope and ownership
Base: 068f592329980f2a17083e1eedc48a6f8a34120c, tree 8771f2f631d842b73aa980c25e15da9093aad8b2, read back from public main; fresh dedicated public clone clean before branch creation.
Branch: codex/ec04-pilot-report.
Sole supervisor/publisher: current parent. Sole file writer: bounded ec04_report_writer worker, dispatched only after this plan is read back. Independent reviewer is read-only and does not repair its review target. These are coordination labels, not authenticated identities.
Owned paths (eight; one unmerged PR):
docs/evaluation-results/ec04-checkpoint-pilot-20261009.md(new dated report).docs/evaluation-cases.md(current evidence link/summary only).docs/evidence-status.md(dated map/link only, retain historical CI).docs/evaluation-fixtures/README.md(current-evidence introductory cross-link only).docs/evaluation-fixtures/interaction.md(same).docs/evaluation-fixtures/code-change.md(same; source preparation stays unrun)..github/scripts/check-product.py(register the report in existing public-document validation, no new execution feature).tests/conformance/test_public_examples.py(existing documentation consistency assertions and focused report guards only).
The current guides and a consistency assertion still say EC01/EC03/EC04 have no trial evidence. Update only the current map to EC01/EC03 and link EC04 separately; preserve the dated EC02 report's historical wording. The checker needs the new report registered so its existing link/privacy checks cover it. These minimal documentation-check changes are part of the publication, not new product behavior, an automatic grader, or an extension of the EC05/EC06-only result validator.
No overlap with another open PR was observed. No change to installed payload, fixture source/input/oracle/rubric bytes or versions, blank templates, behavior, runtime, launcher, schedules, credentials, settings, held Task23/E01/T12 work, support submission, release, Issue closure, final acceptance or merge. No new candidate attempt, collector, trial replay or direct API spending.
Acceptance and verification
- Report exact pack commit/tree, real synthetic baseline versus uncommitted candidate-output identity, input/oracle/source/test/reference bindings, source receipts, attempt/turn/retry counts and all four criterion dispositions.
- Attribute candidate native patch/fingerprint/test records separately from supervisor replay and advisory review. Describe the single pre-frozen checkpoint continuation as an intervention, not autonomous/unguided TDD.
- Preserve the missing complete-action-coverage limit: no authentication, isolation, global non-action, model ranking, installed-kit efficacy or full PASS claims. Unknown actual model/reasoning/client version/token/cost fields stay null, never zero.
- Preserve native turn durations versus supervisor clock observations, including the 33-second polling gap / 3-second supervision deviation, without calling it a candidate violation or inference-latency benchmark.
- Publish bounded summaries and already-public integrity hashes/links only. No private paths, raw commands containing local paths, transcripts, logs, thread IDs, credentials, environment dumps, or raw private archives.
- All six source specifications, fixtures and blank template remain NOT_RUN; input/oracle pending fields are not current trial authority. Previous dated reports and EC05/EC06-only validator remain unchanged.
- Check exact changed-file ownership, link/privacy consistency, source fixture/payload preservation and current report facts against Task #53 receipts. Run the documented local deterministic product/installer checks, relevant public-example/fixture tests and full conformance suite; record unavailable PowerShell/lint tools honestly. Git diff --check and current-head GitHub quality/conformance are required evidence for readiness; no skipped/unavailable result is called passed.
- Independent read-only review verifies exact final diff/HEAD and claims. Publish bounded exact-head evidence before handoff; owner retains final acceptance/merge.
Risk, routing and stop rules
Normal bounded documentation/publication risk, principally overclaiming evidence or disclosing private data. Use the supported public repository and dedicated local clone; preserve the private predecessor checkout. Local source edits use apply_patch. The writer can edit only owned paths and run relevant offline checks, with no GitHub mutations, commits, pushes, new agents or model trials. Parent alone publishes approved GitHub Task/branch/PR artifacts after inspecting the diff. This is an observed local route, not a runtime isolation guarantee.
Stop/replan if new paths, fixture/rubric changes, a new trial or collector, conflicting ownership, current-base drift or new privacy/security assumptions are required. A successful document/check PR does not make the EC04 overall evaluation PASS or establish repository completion.
- Dominant language
- Python
- Stars
- 5
- Forks
- 0
- Avg merge
- 3h 45m
- Merged PRs (30d)
- 29
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from mochan-tk/agentic-dev-kit-for-codex
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
mochan-tk/agentic-dev-kit-for-codex#48 · 5 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 1/100
mochan-tk/agentic-dev-kit-for-codex#56 · 8 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 8/100
mochan-tk/agentic-dev-kit-for-codex#53 · 4 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
mochan-tk/agentic-dev-kit-for-codex#51 · 3 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
mochan-tk/agentic-dev-kit-for-codex#50 · 4 comments ·
Maintainers usually reply within 1 day
All issues in mochan-tk/agentic-dev-kit-for-codex
Similar issues
-
getchaintips leaves out the active tip once a header is staged on it, where Core always lists itOpen
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
btclib-org/btclib-node#2158 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
dbt-msft/dbt-sqlserver#897 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Gathin23/cleanyourtv#2770 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
vllm-project/vllm#64845 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
dekobon/big-code-analysis#1729 ·
Maintainers usually reply within 1 day