Fidelity contract per arm: golden payload diff in CI vs the real harness, starting with Claude Code
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 30/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- github-actions, python
- Domain
- testing-qa
Research direction
Start with dokimi/harnesses/payload_audit.py and its new_leaks function, which already does the marker check, then find where each docker-* arm builds the request it sends to the model. The golden capture has to come from a real Claude Code run, and the skills gap around --plugin-dir and --setting-sources "" is still open. Done means a CI job captures each arm's payload, diffs it against the golden file, and fails on any difference.
Written by the indexing model from the issue text.
Description
Each docker-* arm should send the model what the real harness sends. Today nothing checks that; #351 fixed three leaks found by hand.
- Capture the request per arm (system prompt, tool list, skill listing, prompt) and diff it against a golden capture from the real harness, starting with Claude Code.
- Run it in CI; use
dokimi/harnesses/payload_audit.py(new_leaks) for the marker check, and the golden diff for everything else. - Known gap: skills load via
--plugin-dir, not~/.claude/skills, because the latter is not offered under--setting-sources ""(measured, Claude Code 2.1.286).
- Dominant language
- Python
- Stars
- 9
- Forks
- 9
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 295
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from intel/dokimi
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
intel/dokimi#374 · 2 comments ·
Maintainers usually reply within 1 day
-
repo budget: Copilot profiles get Claude Code's root CLAUDE* rule, so a root AGENTS.md reads offPossibly taken @antonwolfy claimed this today. Open
Difficulty 3/5 Half a day Newbie friendliness 42/100
intel/dokimi#372 · 1 assignee ·
Maintainers usually reply within 1 day
-
novelty: a capped run indexes spans and writes atoms.json from the sample onlyPossibly taken @antonwolfy claimed this today. Open
Difficulty 4/5 3-5 days Newbie friendliness 18/100
intel/dokimi#371 · 1 assignee ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 8/100
Maintainers usually reply within 1 day
Similar issues
-
Paraxial.trace returns NaN for an object at infinity when the entrance pupil is at the first surfaceOpen
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
enhancement ios launcher
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
Substrait consumer doesn't recognise `shift_left` / `shift_right` from functions_arithmetic.yamlOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/datafusion#27059 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
loteran/Arctis-Sound-Manager#368 ·
Maintainers usually reply within 1 day
-
feed submission
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
MobilityData/mobility-database-catalogs#1787 ·
Maintainers usually reply within 2 days