[agentrx-optimizer] Daily Workflow Optimization - 2026-11-12

未关闭 适合新手
  • 哪里不对？
#77,084 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
74/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
github-actions, javascript
领域
ci-cd, observability

调研方向

阅读.github/workflows/auditor.md中约第226–245行、“Collect health failures, cached logs and bounded audits”步骤里的嵌入式 JavaScript；重点关注发现结果的验证和检查点处理。Issue 给出了失败结果，以及对reason: "time_budget"预期的行为。使用gh aw compile或make recompile重新编译，然后检查此部分结果是否会推进或记录检查点，而不是抛出异常；接下来按计划运行的 Auditor 是指定的验证方式。

由索引模型根据 Issue 内容生成。

描述

automation observability optimization traces
Executive Summary

AgentRx analyzed the trailing ~24h of github/gh-aw scheduled-workflow runs (256 runs total; 52 unique failed) using the pre-downloaded log bundle plus one targeted audit call. The largest failure_boundary cluster (as opposed to generic conclusion/"Handle agent failure" aftermath noise) is a 100%-reproducible, self-inflicted crash in the Auditor workflow's failure-evidence collector: every one of the 9 observed scheduled runs in the window failed at the identical step, after the identical ~300s delay, with the identical error. Root cause was traced to the exact source line.

AgentRx Evidence
  • Critical step: Collect failure evidence job → Collect health failures, cached logs and bounded audits step, in workflow Auditor (.github/workflows/auditor.lock.yml, source .github/workflows/auditor.md)
  • Failure category: driver_exit — an unhandled script error from an overly strict result-validation guard, not an agent/LLM reasoning failure
  • Frequency / impact: cluster failure-a7fc54d02b71f423, 9 of 9 observed scheduled "Auditor" runs between 2026-11-11T21:00Z and 2026-11-12T04:28Z failed at this exact step. 8/9 failed at 300-301s (hard time-budget ceiling); 1/9 failed earlier at 203s. This is a failure_boundary signal (the point of failure), distinct from — and much higher-impact than — the generic conclusion/"Handle agent failure" aftermath cluster (22+9 runs, which are downstream symptoms of various upstream failures, not a distinct root cause).
  • Representative run IDs: §43155049221 (fully audited), §43099392497, §43103263034, §43112055596, §43120030767, §43129563405, §43134662425, §43141804956, §43148563539

Exact error (from the audit tool and raw step log of §43155049221):

##[error]Unhandled error: Error: Incomplete or invalid health discovery; checkpoint unchanged: {"partial":true,"reason":"time_budget","failure_listing_complete":false,"failure_pass_complete":false,"workflows_array":true}
AgentRx Artifacts

IR summary: Built trajectory.json directly from compact per-run metadata (run_summary.json/audit output, no raw log text) for 10 runs — the 9 Auditor cluster members plus 1 diversity run (Daily Model Inventory Checker, run §43123321594, a different failure_boundary cluster). The IR stage normally defaults to an LLM-based converter (copilot CLI) for unrecognized domains; that path errored with No authentication information found (no GH_TOKEN/COPILOT_GITHUB_TOKEN in this sandbox). The hand-built IR already matched AgentRx's canonical schema (trajectory_id/instruction/steps[].substeps[].{sub_index,role,content}), so it was seeded directly into trajectory_ir.json and the pipeline resumed from static.

Invariant/checker highlights: static and dynamic invariant generation both require the same LLM endpoint and failed identically (Static invariants JSON parse failed: Expecting value / empty LLM response). Re-ran check with --skip-static --skip-dynamic: it executed deterministically across all 10 trajectories but surfaced 0 violations, because there were no generated invariants to check against — this is an artifact of the missing LLM endpoint, not evidence that the runs were healthy.

Judge classification output: The judge stage (LLM-as-judge root-cause classification) attempted all 10 trajectories but errored after 3 retries per task with Empty response from LLM; only 1/10 tasks produced a (empty) record (num_judges: 0, failures: []). No automated root-cause category was produced.

Known limitations: This sandbox has no copilot CLI authentication, so every LLM-backed AgentRx stage (ir domain-conversion, static, dynamic, judge) is blocked — consistent with the task's documented fallback: continue with completed artifacts (ir, check with empty invariants) and ground the recommendation in direct evidence instead. That direct evidence came from the audit MCP tool's errors[] field plus a targeted grep of the downloaded step log (workflow-logs/5_Collect failure evidence.txt) and the matching source line in .github/workflows/auditor.md.

Classification
violation evidence fix_type rationale
Collect failure evidence job throws a fatal, checkpoint-discarding error whenever gh aw health returns a legitimate partial:true, reason:"time_budget" result 9/9 Auditor runs in window fail identically at ~300s with Incomplete or invalid health discovery; checkpoint unchanged: {"reason":"time_budget",...}; guard code at .github/workflows/auditor.md:234-239 only accepts reason === 'failure_listing_limit' as a recoverable partial state improving retry/backoff strategy The checkpoint/retry mechanism exists precisely to make incremental progress across bounded time budgets, but its allow-list omits the one partial reason (time_budget) that gh aw health's own --timeout 300s flag is documented to produce, turning an expected degraded result into a 100%-reproducible hard failure instead of a checkpointed retry
Generic conclusion/"Handle agent failure" aftermath (clusters failure-e121990520ae929c, failure-4760d85475e78892, 22+9 runs) signal_role: aftermath, spans many unrelated workflows/engines (excluded — symptom, not boundary) Per guidance, aftermath signals describe downstream symptoms of the upstream failures already covered by other failure_boundary clusters; not a distinct, actionable root cause
Recommended Optimization
  • Change: In .github/workflows/auditor.md (~lines 226-245, embedded JS in the "Collect health failures, cached logs and bounded audits" step), extend the discovery-validation guard to also accept discovery.partial === true && discovery.reason === 'time_budget' as a valid partial state (alongside the existing 'failure_listing_limit'), and persist a checkpoint/sampling-gap for whatever was collected before the time budget was hit, instead of throwing an unhandled error that leaves the checkpoint unchanged.
  • Why highest impact: This is the single largest failure_boundary cluster in the window (9 runs, 100% of this workflow's observed scheduled invocations), it's fully reproducible and root-caused to one specific line, and it blocks the Auditor's own purpose (producing fleet health/audit data) — meaning it's actively degrading the same observability pipeline this AgentRx optimizer depends on.
  • Where to implement: .github/workflows/auditor.md lines ~226-245 → recompile to .github/workflows/auditor.lock.yml via gh aw compile / make recompile.
Validation Plan
  • After merging the fix and recompiling the lock file, watch the next several scheduled Auditor runs.
  • Success: the Collect failure evidence job no longer fails with Incomplete or invalid health discovery when gh aw health reports reason:"time_budget"; the step either completes successfully with a saved checkpoint, or at minimum advances/records the checkpoint instead of leaving it unchanged.
  • Expected metric change: Auditor failure rate for this step drops from 9/9 (100%) in a comparable ~7h window to 0, and step durations should stop clustering at exactly 300-301s paired with a hard error.
References

Generated by ⚡ Daily AgentRx Trace Optimizer · claude · agent · ~159 AIC · ⌖ ~1 AIC · ⊞ 4.4K · ◷

  • expires on Nov 18, 2026, 9:47 PM UTC-08:00
主要语言
Go
星标
5.6k
派生
625
平均合并
5 小时 39 分钟
30 天内合并 PR
796

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

  • 提供 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

github/gh-aw 的其他 Issue

查看 github/gh-aw 的全部 Issue

相似的 Issue

更多 Go Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。