Agent harness: rig-based loop, layered prompt system, and failure-driven prompt improvement

Start by reading the proposed build order, especially assemble_brief and syscribe brief, then inspect the existing MCP server tools and example models. The issue proposes a new syscribe-harness crate and several open design decisions, so first scope a single build-order step and identify its existing entry points and tests. Done criteria should come from an ADR, requirements, and PlanningItem tree, which the issue says still need to be created.

Summary

Syscribe already provides the state and governance layer of an agent harness: a PlanningItem work queue, claim/release, an MCP server with guarded writes, [agents] policy (protected_paths, actors), human-owned questions (answerBy: human), an audit trail, and "done" gates (validate, trace-code, E719, W310). What is missing is the agent loop and the prompt system around it.

Proposal: a new syscribe-harness crate in which a rig agent is an MCP client of the existing server, so every guarded-write check still applies, plus a layered prompt system that learns from its own failures.

Goals

1. A loop that takes a PI-*, claims it, assembles context, runs a rig agent, runs the gates, and either marks the item done with evidence or records a question and releases the claim.
2. A layered prompt system whose task context is generated from the model, not hand-written.
3. A feedback loop that turns gate failures into reviewed, evaluated prompt improvements.

Design

Integration shape

• A (recommended first): rig as an MCP client of syscribe mcp. Reuses guarded writes, dry-run/delta/commit, --expect-hash, protected paths.
• B (later, if profiling justifies): depend on syscribe-model and register native rig Tools. Faster, but easy to bypass the guarded-write path.

Prompt layers (stable to volatile, so the prefix caches)

#	Layer	Source
1	Role and rules	new prompts/harness/role.md, distilled from the CLAUDE.md "Safe mutation" rules
2	Format guide	core of create-model.md plus on-demand spec <topic> / help <Type> lookup
3	Project overlay	existing agent_instructions_section hooks (link types, binding time), [agents], [users]
4	Task brief for one PI-*	new assemble_brief(model, pi); syscribe brief <PI> [--json] [--budget N]
5	Gate contract	generated from the real gate commands

Templates live as files in prompts/harness/*.md with a slot delimiter that does not collide with {{X.y}} value references. A [harness] table in .syscribe.toml may override a layer.

The brief contains: the goal and parent/child state (steps), full text of every achieves: requirement with ancestors and breakdownAdr, satisfiedBy elements and implementedBy paths, existing TestCases and verdicts, answered/open questions and blockedBy, and hazards (W311 overlaps, suspect links, protected paths). Each section has a token budget; anything cut is listed as an id pointer.

Learning from failures

1. Run record sidecar per run: prompt layer hashes, model id, PI id, brief hash, outcome (done | gate-failed | escalated | retry-exhausted | abandoned), gate findings per attempt, rejected writes, tool errors, cost.
2. harness failures: deterministic aggregation by finding code and prompt layer, with a cause classification (instruction missing, ignored, brief missing context, unhelpful tool error, model limit).
3. harness analyze: LLM-proposed diff to one layer, emitted as a PlanningItem with a falsifiable claim. Never a direct edit.
4. Replay set and runner: baseline vs candidate, N runs each, gate-pass rate / retries / escalations / cost. One layer changed per proposal. Rules that never fire are pruning candidates.
5. Human gate: approval through an answerBy: human question.

Tool calls the rig agent needs

Most exist in the MCP server today (names from tool_names()); gaps are marked.

Orient and read

Tool	Purpose	Status
get_element	fetch an element (detail, fields projection, docRendered)	exists
steps, questions	PI breakdown, open questions	exists
trace, impact, follow, neighbors	traceability context	exists
search, search_text, list_by_type, children	locate elements	exists
describe_type, explain_finding, template	learn the format on demand	exists
get_brief	the assembled task brief	new

Write (guarded)

Tool	Purpose	Status
next_id	never invent an id	exists
create_element, update_element, move_element, delete_element	model edits	exists
evidence	attach evidence to a PI	exists
question	record a decision instead of guessing	exists
claim_item, release_item	ownership; claim/release are CLI-only today	new (verify)
amend	amendment.add on an accepted ADR / approved requirement	verify via update_element

Verify and gate

Tool	Purpose	Status
validate, validate_element	model checks	exists
trace_code	untraced code units	exists
gates	the "before declaring done" set	exists, check it covers W310/E719
ingest_results	feed test results back as evidence	exists
requirement_coverage	verdict per requirement	exists

Code side (not in the MCP server)

Tool	Purpose	Status
read_file, edit_file	source edits, honouring protected_paths	new
run_command	cargo test, cargo build, allow-listed	new
finish	the agent signals completion, which triggers the gate run	new, harness-local
escalate	raise an answerBy: human question and stop	new, harness-local (wraps question)

finish and escalate are handled by the harness loop itself rather than the model server, so the harness, not the agent, decides when the work is done.

Build order

1. assemble_brief plus syscribe brief, with golden tests on the example models.
2. Embedded layer templates and harness prompt <PI> (print the assembled prompt).
3. Trim create-model.md to a core plus on-demand lookup, measured first.
4. rig loop: claim, brief, agent, gates, retry (capped), escalate, release.
5. Run-record sidecar and harness failures.
6. Replay runner.
7. harness analyze, last.

Open decisions

• Big cached format-guide prefix, or on-demand lookup? (lean: on-demand, then measure)
• Slot delimiter for templates (must not collide with {{X.y}})
• Do prompt files carry amendments: records?
• Retry cap and which PIs an agent may take (exclude answerBy: human blockers)
• Where run records live and whether transcripts are stored (privacy)

Process (per repo convention)

• ADR covering layering, brief, rig-over-MCP, and prompt evolution
• REQ-TRS-HARNESS-* requirements
• Native PlanningItem tree under model/Planning/
• Design element whose implementedBy: names the new crate (avoids W326)
• Discoverability checklist: template, help, search, tasks, explain, agent-guide, docs