Release candidate: independently verify all four Agent Skills in Claude Code and Codex

This is a release-acceptance coordination issue, not a code task: it asks for independent, real-agent verification of four skills in Claude Code and Codex. Start with the skills/*/SKILL.md sources, the plugin manifests and the fixture protocol referenced in #74 and evals/README.md. Done means each skill has recorded positive, negative and hostile trials with unedited traces on both hosts, reviewed by someone other than the implementer.

Owner decision / scope (2026-11-03)

Ship one pinned, installable BeautifulRepo plugin with all four Agent Skills and independently verify real behavior in Claude Code and Codex before calling it ready for external promotion.

Current evidence (refreshed 2026-11-06): the four-skill development candidate, manifest version 0.3.0-dev.0, is pinned to immutable commit cf8ac81ddabb5e2006913e65a2f980ef201baf5c. This development candidate includes the Codex requires_local_executor: true declaration, but its actual installation and runtime behavior remain unverified. The published v0.2.0 plugin contains only audit-repo; master subsequently advanced. Source and prior CI checks exist, but no fresh GitHub-source installation, across-host skill discovery or real model-driven acceptance of all four skills is verified. The old c6e1bae CI reference was a historical source check, not the current candidate's execution evidence. Re-check live CI against the exact candidate for each new run; never transfer a pass from another SHA. Existing #86 is the plugin epic and #74 contains fixture/eval infrastructure.

Test entry point: follow the pinned installer guidance and the candidate's manual matrix. Issue #258 accepts independently reproduced PASS, FAIL and INCONCLUSIVE host traces. Before testing, confirm the actual installed/cached copy and all four SKILL.md files correspond to this commit; a marketplace command completing is not a model pass.

Suggested milestone name: v0.3.0 (four-skill bundle), final version/tag only after release readiness. #255 now has a separate experimental local MCP adapter, but independent host acceptance and any release are still pending. It MUST NOT block or be advertised as part of this four-skill release.

Four candidate workflows — scope and observable contracts

Preparation / packaging acceptance

• Review all four current skills + every referenced file/script for portable relative paths, duplicate/conflicting instructions, unexpected writes, permissions and provider data-leak assumptions.
• Keep ONE source-of-truth skills/<name>/SKILL.md, consistent plugin.json, .claude-plugin/plugin.json, .claude-plugin/marketplace.json, .agents/plugins/marketplace.json; zero phantom tools/MCP/installable CLI claims.
• Ensure all four actual skill directories are inside the chosen pinned candidate tag/commit (not just in master) and each installs/discovers through a fresh public GitHub plugin marketplace, not only a working-tree local copy.
• Lock tested candidate SHA and version across manifests. Stable tag only after acceptance; verify tag resolves to the accepted commit. Retain history and past v0.2.0 plugin support.
• Document Python 3.11+, host/version/OS prerequisites; repeatable install, upgrade, uninstall and restart procedures; example invocations; explicit local tools vs agent-model data-transmission caveat.
• Run complete deterministic repo test suite and plugin-manifest checks on candidate SHA; record CI URL and exact command/exit status. Pass is necessary, NOT sufficient.

Independent test protocol (reuse #74 fixtures, DO NOT build duplicate harness)

Producer prepares a pinned candidate, instructions, test data, expected findings and explicitly marked synthetic fixtures. Independent test executor uses a clean checkout/fresh plugin cache and an isolated disposable synthetic target. Independent reviewer assesses unedited trace, observed file deltas, expected vs actual outcomes and limitations; the producer does not self-accept.

• Claude Code: plugin validation + fresh GitHub-source candidate installation, skill discovery, actual model invocation on synthetic target and readable output. Log client version, OS, installed plugin revision and any unexpected difference from master.
• Codex: independent fresh GitHub-source candidate installation and discovery; actual model invocations using Codex's own runtime (do not assume Claude's eval CLI evaluates Codex). Same pin and fixture, independently recorded.
• Each skill in each host: at least one positive trigger, one unrelated negative trigger, and a task-specific guard/hostile case with observable PASS/FAIL/INCONCLUSIVE. This is 4 × 2 host pairs, not an assumption that one host proves the other.
• Audit: healthy and broken local-link fixture; compare checker output/error statuses to same-revision direct python -I run; read-only snapshot.
• README: unapproved draft must leave target byte-identical; a separate, specifically approved synthetic diff may test an allowed edit and then re-audit; test stale-hash rejection and no unrelated file edits.
• Trust: synthetic disabled security-reporting route and injected fake email; no unsupported public channel or invented license decision.
• Release: tagged, untagged and hostile-subject synthetic Git history; output says Unreleased draft where appropriate, zero tags/changelog/push/publication effects.
• Verify fixture was really visible (actual Read result/path in original trace). Missing visibility, tool not invoked, unavailable sandbox, prompt-only claims or forbidden tool error => INCONCLUSIVE/FAIL as appropriate, never PASS by inference.
• Verify before/after workspace with tools/agent_eval_snapshot.py on disposable fixtures only; record tool calls and effects (a snapshot by itself does not certify no out-of-target effects).
• Secure evidence: scripts and prompts are untrusted; do not run third-party target code or disclose confidential checkout text or tokens in logs/screenshots. Check no unexpected network/credential usage beyond approved model provider.
• Record exact host + plugin SHA + Python/OS versions, task prompt, expected vs actual skill selection, read trace, tool transcript, final output, before/after file hashes, verdict, limitations and reviewer identity. Publish only sanitized, consented artifacts.

Independence requirement: at least one reviewer other than the implementer, and preferably external independent testers for both hosts; explicitly separate internal independent review from externally reproduced. No claims of external acceptance if all runs are by the owner/fleet. External volunteers are invited in the dedicated help wanted issue linked below.

Costs / permissions: model evaluations may incur subscription usage or billed API calls; no automatic eval runs, upgrades, credential use or third-party spending without the operator/tester's informed approval. Do not turn real-agent evals on in normal CI.

Acceptance and release gate

• Each of the four skills is discoverable and actually invoked on an appropriately chosen task in each of Claude Code and Codex from the SAME immutable candidate.
• Source-bound positive and negative/hostile outcomes are independently reviewed; zero critical safety regressions, invented evidence, unapproved target writes or wrong-skill autonomous actions.
• All deterministic tests green on release commit; model tests explicitly graded with unedited evidence and clear unknowns; test coverage/OS limits published honestly.
• README/agent-plugin guide show only tested install syntax and output; documentation clearly distinguishes plugin/skills from the later MCP server.
• After owner-approved release step, create a verified stable release, inspect exact tag tree and provider published release. Until then label everything candidate / development; don't tell external users all four shipped in stable.
• Link official release from launch issue #254 and use as a factual distribution asset. Star metrics and independent users are measured separately, never invented.

Existing work; avoid duplicate issues

• #86 — parent plugin epic
• #74 — existing actual-agent tests, synthetic fixtures and trace protocol
• #70, #71, #72 — skill-specific implementation
• #254 — Starforge product launch readiness
• #255 — distinct MCP workstream, not a prerequisite
• guides/agent-plugin.md, evals/manual-smoke.md, evals/README.md, guides/agent-safety.md

This issue is a versioned release acceptance and evidence coordination item, not an authorization to merge/publish unreviewed implementation or claims that agent tests passed.