[rig-decomposition-bench] Daily rig decomposition benchmark — 2026-10-22 — single-call

writer returned null or whitespace-only source

writer returned null or whitespace-only source

# Ondansetron Injection Shortage Contingency Package

## 1. Analysis: depletion model

**Assumptions:** 1,200 vials on hand, 150 vials/day, no resupply for 42 days. Usage split: ED 90 (60%), Oncology 37.5 (25%), PACU 22.5 (15%). Day 1 is the first day without resupply.

**Baseline (no conservation):** 1,200 / 150 = 8.0. The **stockout occurs on day 8**. Stock reaches zero at the end of day 8, and day 9 is the first uncovered day. The 42-day need is 6,300 vials, so the **supply gap is 5,100 vials**.

**Scenario A, partial conservation (50% cut, 75/day):** stockout on day 16 (1,200 / 75). The gap is 42 x 75 - 1,200 = 1,950 vials. This is not sufficient.

**Scenario B, recommended conservation (28 IV ondansetron vials/day):**
- Maximum sustainable rate = 1,200 / 42 = 28.57/day. The **rationing target is 28 vials/day, an 81% reduction**.
- 28 x 42 = 1,176 vials, leaving a **24-vial reserve at day 42**.
- **There is no stockout** and the supply gap is 0.
- Stock at end of day: day 8 = 976, day 14 = 808, day 28 = 416, day 42 = 24.

| Unit | Baseline/day | Target IV ondansetron/day | Reduction | Where demand goes |
|---|---|---|---|---|
| ED | 90 | 8 | 91% | ODT, then metoclopramide or prochlorperazine |
| Oncology | 37.5 | 15 | 60% | Reserved for highly emetogenic regimens (HEC); the rest uses ODT or granisetron |
| PACU | 22.5 | 5 | 78% | Granisetron first, then other agents |
| **Total** | **150** | **28** | **81%** | |

**Granisetron buffer:** 800 doses / 42 days = at most 19 doses/day. The plan assumes about 15/day, mainly PACU and oncology. This gives 5-HT3 coverage of 28 + 19 = 47 doses/day if needed. ODT, prochlorperazine and metoclopramide are ample.

**Triggers:** if actual use exceeds 28/day for 2 consecutive days, or the on-hand count is below 1,200 - 28 x (day - 1) - 100, move to ondansetron IV for HEC only and use granisetron for the rest.

## 2. Clinical decision rules

**Exclusion rules (applied to all units):**

| Condition | Excludes |
|---|---|
| Amiodarone, or QTc > 470 ms (470 itself is allowed) | ondansetron ODT and IV, granisetron, prochlorperazine (QT-prolonging) |
| Suspected bowel obstruction, or Parkinson's disease | metoclopramide |
| Age > 75 (75 itself is allowed), or seizure disorder | prochlorperazine |
| Cannot take oral | ondansetron ODT |
| HEC oncology regimen | every non-5-HT3 agent (HEC patients keep a 5-HT3 antagonist) |

**Prioritized options per unit (after exclusions, first remaining option is chosen):**
- **ED:** ODT -> metoclopramide -> prochlorperazine -> granisetron -> IV ondansetron (last, reserved).
- **PACU:** ODT (only if oral is possible) -> granisetron -> prochlorperazine -> metoclopramide -> IV ondansetron (last).
- **Oncology, non-HEC:** ODT -> granisetron -> IV ondansetron -> prochlorperazine -> metoclopramide.
- **Oncology, HEC:** IV ondansetron -> granisetron -> ODT (if oral is possible). Nothing else is allowed.

**Escalation:** if no option remains, the result is `ESCALATE` to the clinical pharmacist and the prescriber. For HEC with a QT risk, the oncologist reviews, corrects electrolytes, and considers ECG monitoring.

**Decision table (examples):**

| Patient | Result |
|---|---|
| ED, oral OK, no flags | ODT |
| ED, NPO, no flags | metoclopramide |
| ED, NPO, bowel obstruction, age 75 | prochlorperazine |
| ED, NPO, bowel obstruction, age 76 | granisetron |
| ED, QTc 471, oral OK | metoclopramide |
| ED, NPO, amiodarone, Parkinson's, age 80, seizure | ESCALATE |
| Oncology HEC, amiodarone | ESCALATE (HEC needs a 5-HT3 antagonist, but there is a QT risk) |

## 3. Implementation

```python
ODT = 'ondansetron_ODT_4mg'
OND_IV = 'ondansetron_IV_4mg'
GRAN = 'granisetron_IV_1mg'
PROCH = 'prochlorperazine_IV_10mg'
METO = 'metoclopramide_IV_10mg'
ALL_DRUGS = [ODT, OND_IV, GRAN, PROCH, METO]
FIVE_HT3 = {ODT, OND_IV, GRAN}

PRIORITY = {
    'ED': [ODT, METO, PROCH, GRAN, OND_IV],
    'PACU': [ODT, GRAN, PROCH, METO, OND_IV],
    'ONC': [ODT, GRAN, OND_IV, PROCH, METO],
}
ONC_HEC = [OND_IV, GRAN, ODT]


def select_antiemetic(patient):
    unit = str(patient.get('unit', '')).upper()
    if unit not in PRIORITY:
        raise ValueError('unknown unit: %r' % patient.get('unit'))
    age = patient.get('age')
    if age is None:
        raise ValueError('age is required')
    qtc = patient.get('qtc')
    qt_risk = bool(patient.get('amiodarone')) or (qtc is not None and qtc > 470)
    oral = patient.get('can_take_oral', True)
    hec = bool(patient.get('hec')) and unit == 'ONC'
    order = ONC_HEC if hec else PRIORITY[unit]

    excluded = {}
    for d in ALL_DRUGS:
        r = []
        if hec and d not in order:
            r.append('not a 5-HT3 antagonist (HEC requires one)')
        if qt_risk and d != METO:
            r.append('QT risk (amiodarone or QTc > 470)')
        if d == METO and (patient.get('bowel_obstruction') or patient.get('parkinsons')):
            r.append('metoclopramide contraindicated (obstruction/Parkinson)')
        if d == PROCH and age > 75:
            r.append('prochlorperazine avoided age > 75')
        if d == PROCH and patient.get('seizure'):
            r.append('prochlorperazine avoided with seizure disorder')
        if d == ODT and not oral:
            r.append('cannot take oral')
        if r:
            excluded[d] = r

    viable = [d for d in order if d not in excluded]
    if viable:
        return {'status': 'OK', 'escalate': False, 'primary': viable[0],
                'alternatives': viable[1:], 'excluded': excluded}
    msg = 'No safe option: escalate to clinical pharmacist and prescriber.'
    if hec:
        msg = ('HEC requires a 5-HT3 antagonist but none is safe: oncologist and '
               'pharmacist review (correct electrolytes, consider ECG monitoring).')
    return {'status': 'ESCALATE', 'escalate': True, 'primary': None,
            'alternatives': [], 'excluded': excluded, 'message': msg}
```

```python
import unittest


class TestSelect(unittest.TestCase):
    def test_ed_default_odt(self):
        self.assertEqual(select_antiemetic({'unit': 'ED', 'age': 40})['primary'], ODT)

    def test_ed_npo_metoclopramide(self):
        r = select_antiemetic({'unit': 'ED', 'age': 40, 'can_take_oral': False})
        self.assertEqual(r['primary'], METO)

    def test_qtc_470_not_excluded(self):
        r = select_antiemetic({'unit': 'ED', 'age': 40, 'qtc': 470})
        self.assertEqual(r['primary'], ODT)

    def test_qtc_471_excluded(self):
        r = select_antiemetic({'unit': 'ED', 'age': 40, 'qtc': 471})
        self.assertEqual(r['primary'], METO)
        self.assertIn(ODT, r['excluded'])

    def test_age_75_allowed_proch(self):
        r = select_antiemetic({'unit': 'ED', 'age': 75, 'can_take_oral': False,
                               'bowel_obstruction': True})
        self.assertEqual(r['primary'], PROCH)

    def test_age_76_excludes_proch(self):
        r = select_antiemetic({'unit': 'ED', 'age': 76, 'can_take_oral': False,
                               'bowel_obstruction': True})
        self.assertEqual(r['primary'], GRAN)
        self.assertIn(PROCH, r['excluded'])

    def test_seizure_and_parkinsons_leaves_granisetron(self):
        r = select_antiemetic({'unit': 'ED', 'age': 50, 'can_take_oral': False,
                               'seizure': True, 'parkinsons': True})
        self.assertEqual(r['primary'], GRAN)

    def test_multiple_contraindications_escalate(self):
        r = select_antiemetic({'unit': 'ED', 'age': 80, 'can_take_oral': False,
                               'amiodarone': True, 'parkinsons': True, 'seizure': True})
        self.assertTrue(r['escalate'])
        self.assertEqual(r['status'], 'ESCALATE')
        self.assertIsNone(r['primary'])

    def test_hec_keeps_5ht3(self):
        r = select_antiemetic({'unit': 'ONC', 'age': 60, 'hec': True})
        self.assertEqual(r['primary'], OND_IV)
        self.assertEqual(r['alternatives'][0], GRAN)
        self.assertTrue(all(d in FIVE_HT3 for d in [r['primary']] + r['alternatives']))

    def test_hec_with_amiodarone_escalates(self):
        r = select_antiemetic({'unit': 'ONC', 'age': 60, 'hec': True, 'amiodarone': True})
        self.assertTrue(r['escalate'])
        self.assertIn('HEC', r['message'])

    def test_hec_qtc_470_ok(self):
        r = select_antiemetic({'unit': 'ONC', 'age': 60, 'hec': True, 'qtc': 470})
        self.assertEqual(r['primary'], OND_IV)

    def test_pacu_npo_granisetron(self):
        r = select_antiemetic({'unit': 'PACU', 'age': 50, 'can_take_oral': False})
        self.assertEqual(r['primary'], GRAN)

    def test_onc_nonhec_oral_odt(self):
        self.assertEqual(select_antiemetic({'unit': 'ONC', 'age': 50})['primary'], ODT)

    def test_invalid_unit(self):
        with self.assertRaises(ValueError):
            select_antiemetic({'unit': 'ICU', 'age': 50})


if __name__ == '__main__':
    unittest.main()
```

The 14 tests are hand-traced against the code above. I have not run them in this response, so run `python -m unittest` before use.

## 4. Reviewer pass: consistency checks

| Check | Result |
|---|---|
| Model says day 8; policy and code assume the same supply. | Pass. Day 8 is the last day of stock, and day 9 is the first uncovered day. The wording was clarified. |
| The model's unit targets (8 + 15 + 5 = 28) match the policy, where ED is least likely to use IV ondansetron and oncology HEC gets it first. | Pass. The code puts IV ondansetron last for ED and PACU and first for HEC. |
| The code's granisetron use stays within the 800 doses (at most 19/day). | Pass. The model assumes about 15/day. |
| Boundaries: QTc > 470 and age > 75. | Pass. The code uses strict `>`, and tests cover 470/471 and 75/76. |
| Each stated contraindication is in the code and the decision table. | Pass. |
| Escalation path is documented and tested. | Pass. |

**Discrepancies found and fixed:**
1. **Prochlorperazine and QT:** the first policy draft excluded only 5-HT3 antagonists for QT risk. Prochlorperazine also prolongs QT, so the policy table and the code now exclude it, and only metoclopramide remains.
2. **HEC plus QT risk:** the policy said to keep a 5-HT3 antagonist, while the QT rule would remove them all. Both documents now state that this case returns `ESCALATE` with a specific message, rather than silently dropping the antagonist or silently giving it.
3. **Stockout wording:** "day 8" was ambiguous. It now means stock is depleted at the end of day 8.
4. **Missing inputs:** the code now raises an error for missing age or an unknown unit, instead of guessing.
5. **Oral route:** ODT was offered to NPO patients. The code now excludes it when `can_take_oral` is false.

## 5. One-page summary for the pharmacy director

**Situation:** our sole supplier cannot deliver injectable ondansetron 4 mg for 6 weeks (42 days). We have 1,200 vials and use 150/day.

**Stockout:** with no action, stock runs out at the end of **day 8**, leaving a 5,100-vial gap. Cutting use by half only delays the stockout to day 16.

**Rationing target:** **28 IV ondansetron vials/day house-wide**, which is an 81% reduction. This covers all 42 days with 24 vials spare.
- ED: 8/day
- Oncology: 15/day
- PACU: 5/day

Granisetron (800 doses) is the buffer, at no more than 19/day.

**Key clinical rules:**
- Use oral ODT first wherever the patient can take oral, and use other agents next. IV ondansetron is the last choice in ED and PACU.
- Highly emetogenic oncology regimens always keep a 5-HT3 antagonist, with IV ondansetron first and granisetron second.
- Amiodarone or QTc > 470 ms excludes ondansetron, granisetron and prochlorperazine.
- Suspected bowel obstruction or Parkinson's excludes metoclopramide.
- Age over 75 or a seizure disorder excludes prochlorperazine.
- If no option is safe, the case goes to the pharmacist and prescriber (ESCALATE).

The `select_antiemetic` function encodes these rules and has 14 unit tests.

**Residual risks:**
- Demand may exceed 28/day. A daily count is needed, with the trigger described in section 1.
- Granisetron could also run short, and the supplier could be delayed beyond 6 weeks.
- Patients with an HEC regimen and a QT risk have no automatic option and need manual review.
- Oral ODT is less effective for actively vomiting patients, and metoclopramide and prochlorperazine have side effects, so monitor outcomes.
- Clinicians may need training on the new defaults.
- Ask about alternative sourcing, such as other wholesalers or outsourcing facilities, and the supplier's restock date, and review weekly.

(empty: no decomposed program was generated or executed)

import { agent, workflow, s, repair, configureAgent, copilotEngine } from "rig";

configureAgent(copilotEngine({ server: true }));

const DEADLINE = Date.now() + 25 * 60_000;
const left = () => DEADLINE - Date.now();

const cp = (globalThis as any).process.getBuiltinModule("node:child_process");

type Run = { code: number | null; stdout: string; stderr: string; timedOut: boolean };

function runRig(source: string, flag: string, timeoutMs: number): Promise<Run> {
  return new Promise((resolve) => {
    const child = cp.spawn("node", ["rig.ts", flag], { cwd: (globalThis as any).process.cwd(), stdio: ["pipe", "pipe", "pipe"] });
    let stdout = "";
    let stderr = "";
    let timedOut = false;
    const timer = setTimeout(() => { timedOut = true; child.kill("SIGKILL"); }, timeoutMs);
    child.stdout.on("data", (d: unknown) => { stdout += String(d); });
    child.stderr.on("data", (d: unknown) => { stderr += String(d); });
    child.on("error", (e: Error) => { clearTimeout(timer); resolve({ code: -1, stdout, stderr: stderr + String(e), timedOut }); });
    child.on("close", (code: number | null) => { clearTimeout(timer); resolve({ code, stdout, stderr, timedOut }); });
    child.stdin.on("error", () => {});
    child.stdin.end(source);
  });
}

function stripFences(text: string | null): string {
  if (!text) return "";
  const m = text.match(/```(?:ts|typescript|rig)?\s*\n([\s\S]*?)```/);
  return (m ? m[1] : text).trim();
}

const tail = (t: string, n = 6000) => (t.length > n ? t.slice(-n) : t);

// Agent role: pick a concrete, gradeable, multi-faceted task
const pickTask = agent({
  name: "pickTask",
  model: "small",
  maxTurns: 2,
  addons: [repair()],
  instructions:
    "Pick ONE concrete, complicated task a person or team might face in a single day that would naturally benefit from being split across sub-agents running different models: multi-source research and synthesis, a multi-file coding task with design/implementation/review phases, a structured report combining several independent analyses, or a multi-step data transformation pipeline. It must be self-contained (solvable from its description alone, no external files or live web access) and concrete enough to grade. Vary the domain; seed for variety: " +
    String(Date.now() % 100000) +
    ". Return a title, a domain, a description, and 3-6 concrete, checkable success criteria.",
  output: s.object({
    title: s.string,
    domain: s.string,
    description: s.string,
    criteria: s.array(s.string),
  }),
});

// Agent role: solve the whole task in a single call
const solveSingle = agent({
  name: "solveSingle",
  model: "medium",
  maxTurns: 1,
  output: s.string,
  instructions:
    "Solve the entire task yourself in one response, addressing every success criterion explicitly. No delegation, no tools. Return the complete solution (Markdown and code fences allowed).",
});

const SKELETON = `import { agent, workflow, s } from "rig";

// Agent role: <role>
const analyze = agent({ model: "small", instructions: "...", output: s.string });

// Workflow role: combines the agents' outputs into the final solution
export default workflow({
  meta: { name: "decomposed-solution", description: "..." },
  body: async ({ call }) => {
    const analysis = await call(analyze, "...");
    return { solution: analysis ?? "" };
  },
});`;

const CONTRACT = `Write a self-contained TypeScript "rig" program that solves the task below by splitting it across AT LEAST TWO agents.
Rules:
- Import only from "rig" (agent, workflow, s). No other imports, no console.log.
- Put a "// Agent role: ..." comment directly above each agent() and a "// Workflow role: ..." comment above the workflow().
- Each agent picks model "small" for simple sub-steps and "medium" or "large" for harder ones.
- Every agent producing source text or free-form solution text MUST use output: s.string. Never make an agent re-emit a large solution inside an object wrapper.
- The root must take no required input and be exported with "export default" without being invoked.
- A workflow() spec accepts only meta, an optional input, and body. It has no output or execute field; the final { solution } object is simply what body returns.
- body receives { call, parallel, pipeline, phase, log }; call(agentFn, "string input") returns the output or null on failure. parallel(thunks) takes an array of () => call(...) thunks of the same return type.
- The final answer must COMBINE the agents' outputs (e.g. a synthesis/review agent given the earlier outputs in its prompt string), not come from one agent solving everything.
- Root return value must be an object { solution: string }.
Required shape (abridged to one agent; yours needs at least two):
${SKELETON}
Return ONLY the complete program source, in one \`\`\`ts fence.`;

// Agent role: write the decomposed rig program
const writeProgram = agent({ name: "writeProgram", model: "medium", maxTurns: 1, output: s.string, instructions: CONTRACT });

// Agent role: fix or regenerate the decomposed rig program
const fixProgram = agent({ name: "fixProgram", model: "medium", maxTurns: 1, output: s.string, instructions: CONTRACT });

// Agent role: grade both solutions on content
const grade = agent({
  name: "grade",
  model: "large",
  maxTurns: 1,
  instructions:
    "Score each solution 0-10 on how completely and correctly it satisfies the success criteria. Judge only correctness and completeness of content - not length, and not which approach produced it. Pick winner 'single-call', 'decomposed', or 'tie' and explain the comparison.",
  output: s.object({
    singleScore: s.number,
    decomposedScore: s.number,
    winner: s.enum("single-call", "decomposed", "tie"),
    rationale: s.string,
  }),
});

type Attempt = {
  attempt: number;
  generation: string;
  typecheck: string;
  execute: string;
  error: string;
  note: string;
};

// Workflow role: pick a task, solve it two ways, grade, and return the full report
export default workflow({
  meta: { name: "rig-decomposition-bench", description: "Single-call vs decomposed solution benchmark" },
  body: async ({ call, log }) => {
    const task = await call(pickTask, "Pick the benchmark task.");
    if (!task) {
      return { status: "fail", error: "task picker returned null", task: null, singleMs: 0, decomposedMs: 0, singleSolution: "", decomposedSolution: "", attempts: [] as Attempt[], source: "", grading: null };
    }
    const taskText = `Title: ${task.title}\nDomain: ${task.domain}\nDescription: ${task.description}\nSuccess criteria:\n${task.criteria.map((c, i) => `${i + 1}. ${c}`).join("\n")}`;

    const singlePhase = async () => {
      const t = Date.now();
      const out = await call(solveSingle, taskText, { label: "single", timeout: Math.max(1000, Math.min(8 * 60_000, left())) });
      return { ms: Date.now() - t, out: out ?? "" };
    };

    const decompPhase = async () => {
      const t = Date.now();
      const attempts: Attempt[] = [];
      let source = "";
      let solution = "";
      let prevSource = "";
      let prevError = "";
      for (let n = 1; n <= 2; n++) {
        if (left() < 3 * 60_000) { log("budget exhausted; stopping decomposition"); break; }
        const rec: Attempt = { attempt: n, generation: "", typecheck: "skipped", execute: "skipped", error: "", note: "" };
        let prompt: string;
        let writer = writeProgram;
        if (n === 1) {
          prompt = `Task:\n${taskText}`;
        } else {
          writer = fixProgram;
          prompt = prevSource.trim()
            ? `Task:\n${taskText}\n\nHere is the current program source:\n\`\`\`ts\n${prevSource}\n\`\`\`\n\nIt failed with this exact error:\n${prevError}\n\nFix precisely what the error names while preserving what already worked. Return the complete program.`
            : `Task:\n${taskText}\n\nThe previous attempt returned no source, so there is no source to fix. Generate the complete program from the task and the required skeleton.`;
        }
        const raw = await call(writer, prompt, { label: `writer-${n}`, timeout: Math.max(1000, Math.min(5 * 60_000, left())) });
        const src = stripFences(raw);
        if (!src.trim()) {
          rec.generation = "empty/null";
          prevSource = "";
          prevError = "";
          rec.error = "writer returned null or whitespace-only source";
          attempts.push(rec);
          if (n === 1) attempts[0].note = "next attempt generates the program from scratch";
          continue;
        }
        rec.generation = "ok";
        source = src;
        prevSource = src;
        if (left() < 30_000) { rec.error = "budget exhausted before typecheck"; attempts.push(rec); break; }
        const tc = await runRig(src, "--typecheck", Math.max(1000, Math.min(2 * 60_000, left())));
        if (tc.code !== 0) {
          rec.typecheck = tc.timedOut ? "timeout" : "fail";
          rec.error = tail(`${tc.stdout}\n${tc.stderr}`.trim() || "typecheck failed with no output");
          prevError = rec.error;
          if (n === 1) rec.note = "next attempt asks the fixer to repair the typecheck error";
          attempts.push(rec);
          continue;
        }
        rec.typecheck = "pass";
        if (left() < 60_000) { rec.error = "budget exhausted before execute"; attempts.push(rec); break; }
        const ex = await runRig(src, "--server", Math.max(1000, Math.min(5 * 60_000, left())));
        let parsed = "";
        if (ex.code === 0 && ex.stdout.trim()) {
          try {
            const j = JSON.parse(ex.stdout.trim());
            parsed = typeof j === "string" ? j : typeof j?.solution === "string" ? j.solution : "";
          } catch {
            parsed = ex.stdout.trim();
          }
        }
        if (ex.code === 0 && parsed.trim()) {
          rec.execute = "pass";
          solution = parsed;
          attempts.push(rec);
          break;
        }
        rec.execute = ex.timedOut ? "timeout" : "fail";
        rec.error = tail(`${ex.stderr}\n${ex.stdout}`.trim() || "execution produced no solution");
        prevError = rec.error;
        if (n === 1) rec.note = "next attempt asks the fixer to repair the execute error";
        attempts.push(rec);
      }
      return { ms: Date.now() - t, attempts, source, solution };
    };

    const [single, decomp] = await Promise.all([singlePhase(), decompPhase()]);

    const decomposedForGrade = decomp.solution.trim() ? decomp.solution : "(Decomposition produced no solution: the generated program failed to produce output.)";
    const singleForGrade = single.out.trim() ? single.out : "(Single-call produced no solution.)";
    const grading = left() > 60_000
      ? await call(grade, `${taskText}\n\n=== SOLUTION A (single-call) ===\n${singleForGrade}\n\n=== SOLUTION B (decomposed) ===\n${decomposedForGrade}`, { label: "grade", timeout: Math.max(1000, Math.min(5 * 60_000, left())) })
      : null;

    const last = decomp.attempts[decomp.attempts.length - 1];
    return {
      status: last && last.execute === "pass" ? "pass" : "fail",
      task,
      singleMs: single.ms,
      decomposedMs: decomp.ms,
      singleSolution: single.out,
      decomposedSolution: decomp.solution,
      attempts: decomp.attempts,
      source: decomp.source,
      grading,
      error: "",
    };
  },
});