research: co-optimize FleetSim compression band and pool boundary

Terminology correction

FleetSim's gamma in routing/compress_route.py is a compression-bandwidth ratio: it defines the borderline interval (B_short, gamma * B_short] in which a request may be compressed before choosing a short or long pool.

It is not the runtime gateway prompt-compression ratio. Runtime prompt compression is configured through fields such as max_tokens, min_length, profile, and weights. The unrelated SVM gamma configuration must also remain distinct.

Motivation

Research whether FleetSim can jointly tune its short/long pool threshold and the runtime prompt-compression policy, with a calibrated mapping between the two representations and safeguards against oscillation. C&R means compress-and-route.

Audited upstream baseline (2026-07-15)

• FleetSim has CompressRoute, threshold optimization, B_short, and the compress-and-route (C&R) gamma band.
• The semantic router has a separate prompt-compression implementation/configuration.
• Current main does not define a measured mapping from runtime compression configuration to FleetSim's simulated compression success/quality/cost.
• Runtime capacity/pool actuation remains downstream of semantic routing (#2513).

Research scope

1. Define observable compression outcomes: before/after tokens, latency, failure/no-op, quality/safety evaluation, and policy version.
2. Calibrate a FleetSim model from runtime profiles; do not equate gamma with a runtime compression fraction.
3. Compare independent threshold tuning with joint tuning of:
   • runtime compression profile/max_tokens;
   • FleetSim B_short and C&R gamma;
   • downstream pool capacity recommendation.
4. Add quality/safety loss and uncertainty as hard constraints or explicit objectives.
5. Add hysteresis, cooldown, minimum dwell, confidence thresholds, and rollback.
6. Emit offline/advise-only recommendations first; router recipe changes and pool-control changes require separate owners/approval paths.

Boundaries

• Authorization, safety, output contract, and context limits override compression/capacity objectives.
• No optimization loop runs synchronously in ext_proc.
• No automatic pool mutation from the semantic model selector.
• Simulation parameters and runtime knobs must be versioned and translated explicitly.

Acceptance criteria

• A documented calibration maps runtime observations to FleetSim parameters with error bounds.
• Experiments compare no compression, fixed compression, independent tuning, and joint tuning.
• Quality/safety regressions fail the candidate.
• Hysteresis/rollback prevent oscillation under noisy demand.
• Router and pool recommendations are separately reviewable and auditable.
• Replay records requested compression, actual outcome, recommendation, and downstream action distinctly.
• Results report when joint tuning offers no material benefit.

Likely change surfaces

src/fleet-sim/fleet_sim/routing/compress_route.py, src/fleet-sim/fleet_sim/optimizer/threshold.py, semantic-router prompt-compression/config/ext_proc surfaces, Router Replay, dashboards, docs, and tests.

Related: #2018, #2359, #2513, #2550.

Validation entrypoint

make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"

Follow the reported gates and affected E2E profiles.