[RCA] Add evidence-grounded offline RCA analysis skill

Abierto
#14,237 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
24/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
github, java

Línea de trabajo

Start with the existing skills conventions and inspect the validated intake report, local source/test runner, and sandbox configuration provided by prerequisite #14236. Confirm the agreed evaluation harness and its activation entry point before implementation; the issue says final test paths depend on that merged seam. Done means the offline fixtures produce a cited diagnosis or INSUFFICIENT_EVIDENCE, with sandbox, networking, file-conflict, and evaluation behavior checked by the documented replay command.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

AI Assistant database enhancement

Umbrella: #14222

Blocked by: #14236

Summary

Add evidence-grounded offline RCA analysis skill. Deliver one usable operation, not a framework-only foundation.

Boundary

Proposed activation caller: skills/banyandb-rca-analyze/SKILL.md using validated intake plus a local source checkout. Command names are proposed, not existing APIs.
Reporter collection runs with read-only source access; offline steps use local files only.
No source storage/wire-format changes are requested. The incident format is separately versioned.

Requirements

R1. Require validated intake and identify the exact source/build match per affected component; unknown or mixed builds limit confidence rather than being silently replaced with main.
R2. Produce an incident timeline and competing hypotheses, citing artifact IDs/time ranges and source locations. Separate confirmed facts, plausible causes and missing discriminating evidence.
R3. Use a maintainer-provided, prevalidated sandbox to run maintainer-authored reproductions/tests in an explicitly isolated environment with networking denied and CPU/memory/time limits. Never execute bundle-supplied scripts; report sandbox unavailable rather than pretending verification occurred.
R4. Output a local RCA report with reproduction outcome or INSUFFICIENT_EVIDENCE and the smallest follow-up collection request. No live-database access and no automatic posting.

Acceptance criteria

  • An offline fixture with a known injected failure must cite its causal evidence and reproduce the expected failing assertion against pinned source in the sandbox.
  • A second incomplete fixture must return insufficient evidence rather than the known cause. Embedded prompt injection cannot trigger network access or evidence-supplied execution.
  • Document repeatable skill-evaluation commands and expected report checks; distinguish deterministic harness checks from model-dependent diagnosis quality.
  • Use hand-authored or independently captured fixtures with documented provenance; do not generate expected values using the code under test.
  • Add focused behavioral tests before implementation and record the actual RED command/failure, then GREEN results.
  • Keep existing CLI/API behavior compatible. Never overwrite unrelated user files; output conflicts must be refused.

Scope and readiness audit

  • Boundary and behavior: the single operation above; independently observable output and refusal behavior.
  • Production activation: the named CLI or skill entry point above, included in the same PR.
  • Packages/callers: existing skills conventions; intake report and local source/test runner.
  • Dependencies: #14236 Existing validated sandbox configuration and an agreed skill-evaluation harness are also required.
  • Classification: planned blocked leaf; revalidate prerequisite seams after merge.
  • RED evidence: not executed during this planning task; no claim of passing or failing implementation tests.
  • End-to-end proof: the real command/skill scenarios above, not source-text assertions.
  • Formats/lifecycle/fixtures: private workspace or public archive only; artifact-specific fixture family above.
  • Focused tests: CLI leaves should add TestRCA… coverage under bydbctl/internal/cmd and/or the introduced collector package;
    run go test ./bydbctl/internal/cmd -run TestRCA -count=1 plus the new package tests. Skill leaves must add a replayable evaluation harness
    with an exact command documented in the PR. Final test names/paths are fixed when the prerequisite seam merges.
  • Resource/concurrency checks: cancellation, bounded reads/output and cleanup on failure; use repository panic-aware goroutine wrappers if concurrency is introduced.
  • Decision: one proposed PR. If the named behavior requires an unimplemented server API or another lifecycle change, stop and split that prerequisite.

Required-only scope audit

Offline analysis and sandbox verification are the requested outcome; building a sandbox platform is not.

Removed/deferred: New sandbox infrastructure, automatic historical build downloads, remediation/deployment and multi-model benchmark programs.

Out of scope

Autonomous fix/push/deploy, live diagnosis, arbitrary historical build download, universal automatic root-cause certainty.

Lenguaje dominante
Java
Estrellas
25k
Forks
6.6k
Merge medio
12 h 1 min
PR fusionados (30 d)
43

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/skywalking

Todos los issues de apache/skywalking

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.