[agentrx-optimizer] Daily Workflow Optimization - 2026-11-12

Geschlossen Anfängerfreundlich
  • Stimmt etwas nicht?
#77,084 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Anfängerfreundlichkeit
74/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
github-actions, javascript
Bereich
ci-cd, observability

Rechercherichtung

Lies das eingebettete JavaScript in .github/workflows/auditor.md etwa in den Zeilen 226–245 im Schritt „Collect health failures, cached logs and bounded audits“; konzentriere dich auf die Validierung der Erkennung und die Behandlung von Checkpoints. Das Issue enthält das fehlerhafte Ergebnis und das beabsichtigte Verhalten für reason: "time_budget". Kompiliere mit gh aw compile oder make recompile erneut und prüfe anschließend, ob dieses Teilergebnis den Checkpoint vorrücken lässt oder erfasst, statt einen Fehler auszulösen; die nächsten geplanten Auditor-Läufe sind die angegebene Validierung.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

automation observability optimization traces
Executive Summary

AgentRx analyzed the trailing ~24h of github/gh-aw scheduled-workflow runs (256 runs total; 52 unique failed) using the pre-downloaded log bundle plus one targeted audit call. The largest failure_boundary cluster (as opposed to generic conclusion/"Handle agent failure" aftermath noise) is a 100%-reproducible, self-inflicted crash in the Auditor workflow's failure-evidence collector: every one of the 9 observed scheduled runs in the window failed at the identical step, after the identical ~300s delay, with the identical error. Root cause was traced to the exact source line.

AgentRx Evidence
  • Critical step: Collect failure evidence job → Collect health failures, cached logs and bounded audits step, in workflow Auditor (.github/workflows/auditor.lock.yml, source .github/workflows/auditor.md)
  • Failure category: driver_exit — an unhandled script error from an overly strict result-validation guard, not an agent/LLM reasoning failure
  • Frequency / impact: cluster failure-a7fc54d02b71f423, 9 of 9 observed scheduled "Auditor" runs between 2026-11-11T21:00Z and 2026-11-12T04:28Z failed at this exact step. 8/9 failed at 300-301s (hard time-budget ceiling); 1/9 failed earlier at 203s. This is a failure_boundary signal (the point of failure), distinct from — and much higher-impact than — the generic conclusion/"Handle agent failure" aftermath cluster (22+9 runs, which are downstream symptoms of various upstream failures, not a distinct root cause).
  • Representative run IDs: §43155049221 (fully audited), §43099392497, §43103263034, §43112055596, §43120030767, §43129563405, §43134662425, §43141804956, §43148563539

Exact error (from the audit tool and raw step log of §43155049221):

##[error]Unhandled error: Error: Incomplete or invalid health discovery; checkpoint unchanged: {"partial":true,"reason":"time_budget","failure_listing_complete":false,"failure_pass_complete":false,"workflows_array":true}
AgentRx Artifacts

IR summary: Built trajectory.json directly from compact per-run metadata (run_summary.json/audit output, no raw log text) for 10 runs — the 9 Auditor cluster members plus 1 diversity run (Daily Model Inventory Checker, run §43123321594, a different failure_boundary cluster). The IR stage normally defaults to an LLM-based converter (copilot CLI) for unrecognized domains; that path errored with No authentication information found (no GH_TOKEN/COPILOT_GITHUB_TOKEN in this sandbox). The hand-built IR already matched AgentRx's canonical schema (trajectory_id/instruction/steps[].substeps[].{sub_index,role,content}), so it was seeded directly into trajectory_ir.json and the pipeline resumed from static.

Invariant/checker highlights: static and dynamic invariant generation both require the same LLM endpoint and failed identically (Static invariants JSON parse failed: Expecting value / empty LLM response). Re-ran check with --skip-static --skip-dynamic: it executed deterministically across all 10 trajectories but surfaced 0 violations, because there were no generated invariants to check against — this is an artifact of the missing LLM endpoint, not evidence that the runs were healthy.

Judge classification output: The judge stage (LLM-as-judge root-cause classification) attempted all 10 trajectories but errored after 3 retries per task with Empty response from LLM; only 1/10 tasks produced a (empty) record (num_judges: 0, failures: []). No automated root-cause category was produced.

Known limitations: This sandbox has no copilot CLI authentication, so every LLM-backed AgentRx stage (ir domain-conversion, static, dynamic, judge) is blocked — consistent with the task's documented fallback: continue with completed artifacts (ir, check with empty invariants) and ground the recommendation in direct evidence instead. That direct evidence came from the audit MCP tool's errors[] field plus a targeted grep of the downloaded step log (workflow-logs/5_Collect failure evidence.txt) and the matching source line in .github/workflows/auditor.md.

Classification
violation evidence fix_type rationale
Collect failure evidence job throws a fatal, checkpoint-discarding error whenever gh aw health returns a legitimate partial:true, reason:"time_budget" result 9/9 Auditor runs in window fail identically at ~300s with Incomplete or invalid health discovery; checkpoint unchanged: {"reason":"time_budget",...}; guard code at .github/workflows/auditor.md:234-239 only accepts reason === 'failure_listing_limit' as a recoverable partial state improving retry/backoff strategy The checkpoint/retry mechanism exists precisely to make incremental progress across bounded time budgets, but its allow-list omits the one partial reason (time_budget) that gh aw health's own --timeout 300s flag is documented to produce, turning an expected degraded result into a 100%-reproducible hard failure instead of a checkpointed retry
Generic conclusion/"Handle agent failure" aftermath (clusters failure-e121990520ae929c, failure-4760d85475e78892, 22+9 runs) signal_role: aftermath, spans many unrelated workflows/engines (excluded — symptom, not boundary) Per guidance, aftermath signals describe downstream symptoms of the upstream failures already covered by other failure_boundary clusters; not a distinct, actionable root cause
Recommended Optimization
  • Change: In .github/workflows/auditor.md (~lines 226-245, embedded JS in the "Collect health failures, cached logs and bounded audits" step), extend the discovery-validation guard to also accept discovery.partial === true && discovery.reason === 'time_budget' as a valid partial state (alongside the existing 'failure_listing_limit'), and persist a checkpoint/sampling-gap for whatever was collected before the time budget was hit, instead of throwing an unhandled error that leaves the checkpoint unchanged.
  • Why highest impact: This is the single largest failure_boundary cluster in the window (9 runs, 100% of this workflow's observed scheduled invocations), it's fully reproducible and root-caused to one specific line, and it blocks the Auditor's own purpose (producing fleet health/audit data) — meaning it's actively degrading the same observability pipeline this AgentRx optimizer depends on.
  • Where to implement: .github/workflows/auditor.md lines ~226-245 → recompile to .github/workflows/auditor.lock.yml via gh aw compile / make recompile.
Validation Plan
  • After merging the fix and recompiling the lock file, watch the next several scheduled Auditor runs.
  • Success: the Collect failure evidence job no longer fails with Incomplete or invalid health discovery when gh aw health reports reason:"time_budget"; the step either completes successfully with a saved checkpoint, or at minimum advances/records the checkpoint instead of leaving it unchanged.
  • Expected metric change: Auditor failure rate for this step drops from 9/9 (100%) in a comparable ~7h window to 0, and step durations should stop clustering at exactly 300-301s paired with a hard error.
References

Generated by ⚡ Daily AgentRx Trace Optimizer · claude · agent · ~159 AIC · ⌖ ~1 AIC · ⊞ 4.4K · ◷

  • expires on Nov 18, 2026, 9:47 PM UTC-08:00
Vorherrschende Sprache
Go
Sterne
5.6k
Forks
625
Ø Merge
5 Std. 42 Min.
Gemergte PRs (30 T.)
796

Entwicklungsumgebung

In Codespaces öffnen

Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus github/gh-aw

Alle Issues in github/gh-aw

Ähnliche Issues

Weitere Issues zu Go

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.