[agentrx-optimizer] Daily Workflow Optimization - 2026-11-12

Aperta Adatta ai principianti
  • Qualcosa non va?
#77,084 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
74/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
github-actions, javascript
Ambito
ci-cd, observability

Direzione di ricerca

Leggi il JavaScript incorporato in .github/workflows/auditor.md, intorno alle righe 226–245, nel passaggio “Collect health failures, cached logs and bounded audits”; concentrati sulla convalida del rilevamento e sulla gestione dei checkpoint. L’issue riporta il risultato non riuscito e il comportamento previsto per reason: "time_budget". Ricompila con gh aw compile o make recompile, quindi verifica che questo risultato parziale faccia avanzare o registri il checkpoint invece di generare un errore; le successive esecuzioni pianificate di Auditor sono la convalida indicata.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

automation observability optimization traces
Executive Summary

AgentRx analyzed the trailing ~24h of github/gh-aw scheduled-workflow runs (256 runs total; 52 unique failed) using the pre-downloaded log bundle plus one targeted audit call. The largest failure_boundary cluster (as opposed to generic conclusion/"Handle agent failure" aftermath noise) is a 100%-reproducible, self-inflicted crash in the Auditor workflow's failure-evidence collector: every one of the 9 observed scheduled runs in the window failed at the identical step, after the identical ~300s delay, with the identical error. Root cause was traced to the exact source line.

AgentRx Evidence
  • Critical step: Collect failure evidence job → Collect health failures, cached logs and bounded audits step, in workflow Auditor (.github/workflows/auditor.lock.yml, source .github/workflows/auditor.md)
  • Failure category: driver_exit — an unhandled script error from an overly strict result-validation guard, not an agent/LLM reasoning failure
  • Frequency / impact: cluster failure-a7fc54d02b71f423, 9 of 9 observed scheduled "Auditor" runs between 2026-11-11T21:00Z and 2026-11-12T04:28Z failed at this exact step. 8/9 failed at 300-301s (hard time-budget ceiling); 1/9 failed earlier at 203s. This is a failure_boundary signal (the point of failure), distinct from — and much higher-impact than — the generic conclusion/"Handle agent failure" aftermath cluster (22+9 runs, which are downstream symptoms of various upstream failures, not a distinct root cause).
  • Representative run IDs: §43155049221 (fully audited), §43099392497, §43103263034, §43112055596, §43120030767, §43129563405, §43134662425, §43141804956, §43148563539

Exact error (from the audit tool and raw step log of §43155049221):

##[error]Unhandled error: Error: Incomplete or invalid health discovery; checkpoint unchanged: {"partial":true,"reason":"time_budget","failure_listing_complete":false,"failure_pass_complete":false,"workflows_array":true}
AgentRx Artifacts

IR summary: Built trajectory.json directly from compact per-run metadata (run_summary.json/audit output, no raw log text) for 10 runs — the 9 Auditor cluster members plus 1 diversity run (Daily Model Inventory Checker, run §43123321594, a different failure_boundary cluster). The IR stage normally defaults to an LLM-based converter (copilot CLI) for unrecognized domains; that path errored with No authentication information found (no GH_TOKEN/COPILOT_GITHUB_TOKEN in this sandbox). The hand-built IR already matched AgentRx's canonical schema (trajectory_id/instruction/steps[].substeps[].{sub_index,role,content}), so it was seeded directly into trajectory_ir.json and the pipeline resumed from static.

Invariant/checker highlights: static and dynamic invariant generation both require the same LLM endpoint and failed identically (Static invariants JSON parse failed: Expecting value / empty LLM response). Re-ran check with --skip-static --skip-dynamic: it executed deterministically across all 10 trajectories but surfaced 0 violations, because there were no generated invariants to check against — this is an artifact of the missing LLM endpoint, not evidence that the runs were healthy.

Judge classification output: The judge stage (LLM-as-judge root-cause classification) attempted all 10 trajectories but errored after 3 retries per task with Empty response from LLM; only 1/10 tasks produced a (empty) record (num_judges: 0, failures: []). No automated root-cause category was produced.

Known limitations: This sandbox has no copilot CLI authentication, so every LLM-backed AgentRx stage (ir domain-conversion, static, dynamic, judge) is blocked — consistent with the task's documented fallback: continue with completed artifacts (ir, check with empty invariants) and ground the recommendation in direct evidence instead. That direct evidence came from the audit MCP tool's errors[] field plus a targeted grep of the downloaded step log (workflow-logs/5_Collect failure evidence.txt) and the matching source line in .github/workflows/auditor.md.

Classification
violation evidence fix_type rationale
Collect failure evidence job throws a fatal, checkpoint-discarding error whenever gh aw health returns a legitimate partial:true, reason:"time_budget" result 9/9 Auditor runs in window fail identically at ~300s with Incomplete or invalid health discovery; checkpoint unchanged: {"reason":"time_budget",...}; guard code at .github/workflows/auditor.md:234-239 only accepts reason === 'failure_listing_limit' as a recoverable partial state improving retry/backoff strategy The checkpoint/retry mechanism exists precisely to make incremental progress across bounded time budgets, but its allow-list omits the one partial reason (time_budget) that gh aw health's own --timeout 300s flag is documented to produce, turning an expected degraded result into a 100%-reproducible hard failure instead of a checkpointed retry
Generic conclusion/"Handle agent failure" aftermath (clusters failure-e121990520ae929c, failure-4760d85475e78892, 22+9 runs) signal_role: aftermath, spans many unrelated workflows/engines (excluded — symptom, not boundary) Per guidance, aftermath signals describe downstream symptoms of the upstream failures already covered by other failure_boundary clusters; not a distinct, actionable root cause
Recommended Optimization
  • Change: In .github/workflows/auditor.md (~lines 226-245, embedded JS in the "Collect health failures, cached logs and bounded audits" step), extend the discovery-validation guard to also accept discovery.partial === true && discovery.reason === 'time_budget' as a valid partial state (alongside the existing 'failure_listing_limit'), and persist a checkpoint/sampling-gap for whatever was collected before the time budget was hit, instead of throwing an unhandled error that leaves the checkpoint unchanged.
  • Why highest impact: This is the single largest failure_boundary cluster in the window (9 runs, 100% of this workflow's observed scheduled invocations), it's fully reproducible and root-caused to one specific line, and it blocks the Auditor's own purpose (producing fleet health/audit data) — meaning it's actively degrading the same observability pipeline this AgentRx optimizer depends on.
  • Where to implement: .github/workflows/auditor.md lines ~226-245 → recompile to .github/workflows/auditor.lock.yml via gh aw compile / make recompile.
Validation Plan
  • After merging the fix and recompiling the lock file, watch the next several scheduled Auditor runs.
  • Success: the Collect failure evidence job no longer fails with Incomplete or invalid health discovery when gh aw health reports reason:"time_budget"; the step either completes successfully with a saved checkpoint, or at minimum advances/records the checkpoint instead of leaving it unchanged.
  • Expected metric change: Auditor failure rate for this step drops from 9/9 (100%) in a comparable ~7h window to 0, and step durations should stop clustering at exactly 300-301s paired with a hard error.
References

Generated by ⚡ Daily AgentRx Trace Optimizer · claude · agent · ~159 AIC · ⌖ ~1 AIC · ⊞ 4.4K · ◷

  • expires on Nov 18, 2026, 9:47 PM UTC-08:00
Lingua principale
Go
Stelle
5.6k
Fork
625
Merge medio
5h 39m
PR unite (30g)
796

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di github/gh-aw

Tutte le issue di github/gh-aw

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.