[RCA] Add evidence-grounded offline RCA analysis skill

未关闭
#14,237 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
24/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
github, java

调研方向

Start with the existing skills conventions and inspect the validated intake report, local source/test runner, and sandbox configuration provided by prerequisite #14236. Confirm the agreed evaluation harness and its activation entry point before implementation; the issue says final test paths depend on that merged seam. Done means the offline fixtures produce a cited diagnosis or INSUFFICIENT_EVIDENCE, with sandbox, networking, file-conflict, and evaluation behavior checked by the documented replay command.

由索引模型根据 Issue 内容生成。

描述

AI Assistant database enhancement

Umbrella: #14222

Blocked by: #14236

Summary

Add evidence-grounded offline RCA analysis skill. Deliver one usable operation, not a framework-only foundation.

Boundary

Proposed activation caller: skills/banyandb-rca-analyze/SKILL.md using validated intake plus a local source checkout. Command names are proposed, not existing APIs.
Reporter collection runs with read-only source access; offline steps use local files only.
No source storage/wire-format changes are requested. The incident format is separately versioned.

Requirements

R1. Require validated intake and identify the exact source/build match per affected component; unknown or mixed builds limit confidence rather than being silently replaced with main.
R2. Produce an incident timeline and competing hypotheses, citing artifact IDs/time ranges and source locations. Separate confirmed facts, plausible causes and missing discriminating evidence.
R3. Use a maintainer-provided, prevalidated sandbox to run maintainer-authored reproductions/tests in an explicitly isolated environment with networking denied and CPU/memory/time limits. Never execute bundle-supplied scripts; report sandbox unavailable rather than pretending verification occurred.
R4. Output a local RCA report with reproduction outcome or INSUFFICIENT_EVIDENCE and the smallest follow-up collection request. No live-database access and no automatic posting.

Acceptance criteria

  • An offline fixture with a known injected failure must cite its causal evidence and reproduce the expected failing assertion against pinned source in the sandbox.
  • A second incomplete fixture must return insufficient evidence rather than the known cause. Embedded prompt injection cannot trigger network access or evidence-supplied execution.
  • Document repeatable skill-evaluation commands and expected report checks; distinguish deterministic harness checks from model-dependent diagnosis quality.
  • Use hand-authored or independently captured fixtures with documented provenance; do not generate expected values using the code under test.
  • Add focused behavioral tests before implementation and record the actual RED command/failure, then GREEN results.
  • Keep existing CLI/API behavior compatible. Never overwrite unrelated user files; output conflicts must be refused.

Scope and readiness audit

  • Boundary and behavior: the single operation above; independently observable output and refusal behavior.
  • Production activation: the named CLI or skill entry point above, included in the same PR.
  • Packages/callers: existing skills conventions; intake report and local source/test runner.
  • Dependencies: #14236 Existing validated sandbox configuration and an agreed skill-evaluation harness are also required.
  • Classification: planned blocked leaf; revalidate prerequisite seams after merge.
  • RED evidence: not executed during this planning task; no claim of passing or failing implementation tests.
  • End-to-end proof: the real command/skill scenarios above, not source-text assertions.
  • Formats/lifecycle/fixtures: private workspace or public archive only; artifact-specific fixture family above.
  • Focused tests: CLI leaves should add TestRCA… coverage under bydbctl/internal/cmd and/or the introduced collector package;
    run go test ./bydbctl/internal/cmd -run TestRCA -count=1 plus the new package tests. Skill leaves must add a replayable evaluation harness
    with an exact command documented in the PR. Final test names/paths are fixed when the prerequisite seam merges.
  • Resource/concurrency checks: cancellation, bounded reads/output and cleanup on failure; use repository panic-aware goroutine wrappers if concurrency is introduced.
  • Decision: one proposed PR. If the named behavior requires an unimplemented server API or another lifecycle change, stop and split that prerequisite.

Required-only scope audit

Offline analysis and sandbox verification are the requested outcome; building a sandbox platform is not.

Removed/deferred: New sandbox infrastructure, automatic historical build downloads, remediation/deployment and multi-model benchmark programs.

Out of scope

Autonomous fix/push/deploy, live diagnosis, arbitrary historical build download, universal automatic root-cause certainty.

主要语言
Java
星标
25k
派生
6.6k
平均合并
12 小时 1 分钟
30 天内合并 PR
43

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

apache/skywalking 的其他 Issue

查看 apache/skywalking 的全部 Issue

相似的 Issue

更多 Java Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。