[P2][TOOLING] Produce reproducible agent validation bundles
Maintainers usually reply within 1 day
@bozoslav is already working on this.
Since Nov 11, 2026.
Assessment
This issue has not been assessed yet.
Description
Problem and outcome
Give a reviewer one inspectable artifact connecting a proposed fix to build identity, reproducible steps and observed outcomes.
Verified starting point
crates/crabby_mcp/src/lib.rs already offers launch, read_logs, screenshot, run_ticks, watch and performance tools; src/control.rs records engine version and executable/cwd identity. There is no integrated evidence-bundle workflow in these sources.
Bounded scope
Build a thin client-side orchestration example/utility over existing tools. A versioned manifest records commit/build/features when supplied, scenario identity/seed, commands, assertion outcomes, logs, screenshots and hashes/paths of collected artifacts. Mark unavailable metrics and headless screenshots explicitly. Include a deliberate failure followed by a corrected successful run.
Acceptance criteria
- One command creates a self-contained bundle for a small reference interaction.
- Failed build, launch timeout and failed assertion produce useful bounded failure artifacts.
- Secrets/control tokens are excluded and user paths can be redacted without destroying reproduction metadata.
- Repeated run comparisons identify build/settings differences; visual evidence requires inspected images and does not assert universal pixel identity.
Prerequisites and coordination
Depends on #179. Rust-only implementation contributions are welcome; maintainer-owned guide/native interface follow-ups should be named and tracked when applicable. Preserve existing Rust, ABI, save and MCP contracts.
Non-goals
No new profiling subsystem: reuse existing reports and later #145/#153/#157 outputs. No automatic claim that screenshots prove correctness.
First contribution
Implement a thin orchestration example that records build identity, scenario result and logs; add optional screenshots/performance artifacts afterward.
Planning constraints
Preserve the integrated Game/Ctx API and existing Rust/C/C++/save/MCP compatibility. Internal modules are the default; justify any new crate, trait, dependency or indirection with a concrete use case and measured or testable benefit. No automatic full-subsystem replacement or mandatory plugin framework. This is planned work, not a claim of delivery or a release-date promise.
Implementation prerequisites
- #179
- Dominant language
- Rust
- Stars
- 4
- Forks
- 0
- Avg merge
- 45m
- Merged PRs (30d)
- 199
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ponkoengine/ponko
-
area: ecs bug priority: P2 type: testing
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ponkoengine/ponko#321 ·
Maintainers usually reply within 1 day
-
area: rendering bug priority: P2 type: testing
Difficulty 4/5 3-5 days Newbie friendliness 44/100
ponkoengine/ponko#322 ·
Maintainers usually reply within 1 day
-
area: build area: platforms help wanted priority: P3 type: feature
Difficulty 4/5 3-5 days Newbie friendliness 45/100
ponkoengine/ponko#282 ·
Maintainers usually reply within 1 day
-
area: tooling priority: P2 type: feature
Difficulty 5/5 Over a week Newbie friendliness 35/100
ponkoengine/ponko#262 ·
Maintainers usually reply within 1 day
-
area: build documentation help wanted priority: P2 type: feature
Difficulty 4/5 3-5 days Newbie friendliness 45/100
ponkoengine/ponko#246 ·
Maintainers usually reply within 1 day
All issues in ponkoengine/ponko
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
design-data-spec
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
adobe/spectrum-design-data#1744 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
USERID reads /etc/passwd directly instead of looking the user upPossibly taken A pull request linked to this issue is open or already merged. Opendefect
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day