feat(ci): token budget ratchet — first slice over agent eager schemas and server instructions

Start in internal/mcp: read serverinstructions_length_test.go (the 2539→2048 truncation ceiling test being consolidated) and the server construction path around NewServerWithConfig with the agent allowlist, plus how tools/list output is marshalled by the mark3labs/mcp-go SDK. First task is token_budget_test.go alongside those tests, writing rune counts into internal/mcp/testdata/token_budgets.json under an -update flag, then asserting measured <= budget in CI; done means a pinned baseline, the ceiling kept as a hard backstop, and a companion assertion failing for any agent-profile tool missing a budget entry.

📋 Pre-flight Checks

• I have searched existing issues and this is not a duplicate
• I understand this issue needs status:approved before a PR can be opened

Revised 2026-09-24 per maintainer feedback: narrowed to a first slice over two stable, high-cost surfaces. Responses, hook prose, and plugin payloads become candidates for later measured additions, each landing as its own follow-up once the harness exists.

🔍 Problem Description

Engram's context cost is invisible and unmeasured. Two surfaces dominate the per-session bill and have no ratchet today:

1. Eager tool schemas (agent profile) — every agent session receives the full tools/list payload for ProfileAgent at handshake, before any call happens. Today a field added to a response schema, a paragraph added to a tool description, or a tool moved into the agent profile costs a few hundred runes permanently, per session, with no CI signal.
2. Server instructions (agent profile) — a serverInstructions constant with a documented failure class: Claude Code truncates the handshake instructions field at 2048 runes and mem_* tools stopped responding shortly after connect in the field reports (serverinstructions_length_test.go documents the empirical 2539 → 2048 truncation). Today only a ceiling test exists; it can never shrink, so any growth up to the cliff is free.

Both surfaces are pure functions of the code (no DB fixture, no runtime state), which is what makes them the right first slice: deterministic to regenerate, immune to fixture churn.

💡 Proposed Solution (first slice: agent eager schemas + server instructions)

Budget file — internal/mcp/testdata/token_budgets.json, rune counts (the client's currency, same rule as the existing truncation test):

{
  "eager_schemas.agent": <measured at first pin>,
  "server_instructions.agent": <measured at first pin>
}

Test — token_budget_test.go in internal/mcp, next to the surfaces it measures:

• eager_schemas.agent: build the server via NewServerWithConfig with the agent allowlist, invoke the tools/list result the SDK produces, marshal once, count runes. The first go test -run TestTokenBudget -update run pins the baseline; from then on CI asserts measured <= budget.
• server_instructions.agent: same measurement for the instructions constant, consolidating the existing 2048-rune ceiling test into the same file. The ratchet baseline replaces the "free growth up to the cliff" behavior; the ceiling stays as a hard backstop above the budget.

Reproducible baseline — the test is the reproduction: -update regenerates from code alone (no LLM, no tokenizer, no manual audit, no network). The manual-audit figures quoted in perf backlog items (#1040, #1041) get replaced by this pinned number at first landing.

User-facing regressions this catches:

• A tool added to ProfileAgent, a schema field, or a description paragraph inflating every agent session's handshake cost (the #163 class, found manually last time).
• Instructions growth heading back toward the documented 2048-rune truncation cliff where mem_* tools stopped responding.

When a budget increase is legitimate:

• Decreases are always free.
• An increase is legitimate only when the PR body names the capability that grew (new agent tool, new required field), states the per-session rune delta, and the maintainer acks it explicitly. Rewording, extra examples, or restating the same capability do not qualify; those land as decreases or no-change.
• SDK bumps (mark3labs/mcp-go in go.mod) that change serialization show up as a single reviewable budget diff — that is signal, not noise: the client really pays the new size.

Avoiding false positives on harmless changes:

• Only code-pure surfaces are measured in this slice: no fixture DB, no golden query data, no timestamps or IDs inside the payloads. Fixture churn is structurally impossible here; it is the reason responses/hooks/plugins are not in slice one.
• Rune counting matches the client decode rule already established by the truncation test, so byte/rune drift never trips the gate.
• A companion assertion fails when a tool is registered in the agent profile without a budget entry, so nothing silently opts out of the ratchet.

Later additions (candidates, not requirements): response.mem_search.* golden queries, the ToolSearch hook prose from #1077, and plugin static payloads (pi, opencode, claude-code), each as its own measured budget line once slice one's harness lands.

Update flow — -update rewrites the JSON; increases gated by the rule above. Perf PRs demonstrate wins as budget-file diffs (e.g. 11700 → 11060) instead of hand-computed estimates, and the perf-check make target family gains a token-budget-check sibling.

📦 Affected Area

MCP Server (tools, transport) — internal/mcp tests + testdata only for this slice.

🔄 Alternatives Considered

• Full-suite ratchet from day one: rejected per maintainer feedback; cross-package fixture and maintenance contract before the first value lands.
• Keep only the ceiling test: covers instructions alone and never shrinks; schemas (the larger surface) stays invisible.
• Manual audits (status quo): unverifiable, stale immediately, what produced #163's late discovery.
• Real tokenizer in CI: heavyweight dependency for precision a ratchet does not need; rune counts rank and regress-detect identically for gating purposes.

📎 Additional Context

Completes the cost-coverage matrix direction started by #897 (latency) and #974 (dead code) with the first context-cost gate: latency, dead code, context. Aligns with the executable-invariants direction of #872.