Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[REC-AR6][P0] Deterministic Recovery MVP Evaluation and Go/No-Go (Advisor Optional)

Abierto
  • ¿Algo no está bien?
#375 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
ai-infra-agents, security, testing-qa

Línea de trabajo

Start by reviewing the REC-AR0..AR5 dependency issues and the security gate issues #318, #323, and #306. Define reproducible fake-E2E and adversarial evaluations covering the listed recovery cases, fixed models and budgets, crash/restart, offline effects, and independently verified outcomes. Done means tests preserve existing BLOCKED behavior and authority gates, report a go/no-go, and never run paid evaluations without explicit approval.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Parent: #354 · Stage: MVP M2 · Priority: P0 (deterministic evaluation) · Do not gate on optional Advisor issues.

Goal

Evaluate the existing Controller + #369 typed stop reasons + #374 deterministic escalation package against current operator experience. The EPIC can close its deterministic MVP with explicit evidence even if both #370 Advisor gates are no-go.

Study and delivery

  1. Preregister an offline/evidence review baseline including the review report's reported 51 runs / 15 unblocks (not independently verified here), fixed #291/#327 regressions, true K9/K10/manual coordination, crash recovery, LOCAL blocked, FAILED, unchanged-phase errors, timeouts and non-AutoForge samples where available.
  2. Define gold-standard operator resolution using Controller live readback or separately reviewed human labels, distinguish actual automated recovery from merely providing guidance, and never count Agent/Advisor confidence as outcome proof.
  3. Compare original status/errors against deterministic typed classification + action package. Measure classification accuracy/coverage, actionable handoff coverage, incorrect/unsafe suggested actions, human diagnosis time/steps, repeat BLOCKs, preserved candidates and effects, latency and added runtime cost. Include uncertainty, missing samples and selection bias.
  4. Required security regressions: zero unauthorized GitHub writes/unblock/approvals; zero lost unpushed commits or lost pending effect receipts; no bypass of exact-candidate validation, ReviewCertificate, CI, merge/ownership locks; no altered default Controller decisions; crash and JSON backward-compatibility tests.
  5. Record independent deterministic MVP go/no-go and operator sign-off. Advisor diagnosis shadow comparisons, if Gate D approves, belong to a separate follow-up evaluation Issue, not a blocker of this task. Autonomous execution, if Gate A approves, likewise requires separately reviewed safety rollout.
  6. No real paid-model calls, changed CI thresholds or autoforge epic run merely for planning. Default-on changes require their own approval even if this evaluation passes.

Acceptance criteria

  • Reproducible and annotated before/after matrix showing objective handoff effectiveness, not only automated unblock percentage.
  • Safety/correctness and crash/unknown-effect regression tests pass; unknown/ambiguous stays explicit.
  • #370 Gate D and A decisions recorded (including no-go), with optional #371–#373 not required for MVP closure.
  • Maintainer-approved roll-forward/rollback or explicit no-go with remaining risks and docs/status guidance.

Dependencies

Hard: #369, #370 and #374. No hard dependency on #371, #372 or #373, and no blanket #315/#319/#320/#321 gate. Coordinate #318/#323 evaluation conventions and #306 security boundaries.

Non-negotiable invariants

  • Controller-only authority for phase/task transitions, typed GitHub effects, candidate/evidence identity, authorization, validation, ReviewCertificate and merge; operator-only K9/K10 semantics unchanged. Existing BLOCKED and FAILED exit rules unchanged, LOCAL never gains an unblock edge by implication.
  • No provider receives GitHub mutation credentials, no prompt/source text is authorization, no untrusted shell commands outside existing sandbox policy. Preserve unpushed FIX candidates, effect journals and original evidence; reconcile UNKNOWN outcomes before retrying.
  • No implementation or paid-model benchmark is authorized by this planning change; default single-agent behavior unchanged. Persisted schema migrations require coordination on #23, using the protocol current at implementation time rather than reserving a number.
  • Native GitHub links remain to be established and read back: prose dependencies are not scheduler-enforceable blocked_by edges.

Review rationale: #354 review report.

Lenguaje dominante
Python
Estrellas
0
Forks
0
Merge medio
5 h 16 min
PR fusionados (30 d)
100

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de iceboundrock/AutoForge

Todos los issues de iceboundrock/AutoForge

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.