Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

EPIC: Adaptive Intra-Issue Deliberation and Evidence Ledger (GVS5H-inspired)

Open
#315 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
15/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Active
Tech stack
github, python, sqlite
Domain
ai, backend

Research direction

The EPIC plans intra-issue deliberation; start by reading ADR 0004-0006 (#158) and #176 for authority and scheduler contracts, then inspect ControllerEngine and loop_guard.py in the Python codebase at base f4fd5ef. The first concrete step is linking child issues #316-#326 as native GitHub sub-issues with blocked_by edges via API, then verify with autoforge epic status before any run.

Written by the indexing model from the issue text.

Description

enhancement

Type: standalone architecture, implementation and evaluation EPIC (planning only). Research inspiration: GVS5H paper, source scaffold. Planning baseline: main at f4fd5efec6d5ffbf12397a0ba59560eeb2379148 (2026-10-25 UTC). No code or providers were invoked for this planning action.

Objective

Introduce opt-in, bounded, evidence-driven deliberation within one GitHub Issue: concise durable context/ledger, fresh-context workers, independent verification feedback and adaptive orchestration when it measurably improves successful validated PRs or cost-to-success. Default remains the existing single-agent workflow. Do not hardcode multi-agent as an unconditional quality improvement.

GVS5H demonstrates a sequential manager/worker method (not five concurrent edits), disk-curated plan/notes, fresh calls, public-sample verification and no-progress guards on algorithmic LiveCodeBench. Results do not establish superiority for real repositories, integration tests, GitHub workflows, heterogeneous models, or a stronger single-agent baseline with tools. Research code is an experiment, not the production execution engine.

Architecture / ownership (non-negotiable)

  • ControllerEngine owns all authority: canonical ANALYZE_EXECUTE → REVIEW → FIX → READY_FOR_MERGE state transitions, step/attempt budgets, Git/PR/issue effects, candidate verification and publication, ReviewCertificates and MERGE gates. A model-generated done, task list or notes never establishes acceptance.
  • EPIC scheduler #176 owns cross-issue scheduling and native GitHub subIssues/blocked_by edges. This EPIC owns intra-issue ephemeral planning. Its internal tasks must NEVER be treated as native GitHub issues, mutate the EPIC graph, or alter scheduler locks L0/L2/L1.
  • Existing AgentProvider interfaces (Claude Code CLI, OpenCode 2.x CLI, Pi RPC; eventual SDKs under #147) execute bounded scoped work. Manager / Worker are roles, not new providers or privileged subprocesses. Preserve resource/auth/isolation rules including #266; no agent gets gh, GitHub write credentials or sandbox-escape privileges.
  • Ledger is controller-owned, scoped to repo/issue/run/phase/base/candidate SHA and typed, bounded, redacted. Plan, findings, attempts and curated handoff are untrusted advice/data, not executable configuration, GitHub effect requests, review evidence or workflow authority. Prompt-injection/repo-content, state tampering, replay, concurrency and migration must be threat-modelled; gate resume on identity/candidate compatibility.
  • Tests and verifiers use existing executor/sandbox launch classes, not a new unsandboxed host shell. Programmatic negative verdicts block model-declared completion; positive sample/test results do not replace independent review, validation or merge checks.
  • Crash safety and cost safety: at most one active mutating Worker for an issue initially. Ledger writes and candidate updates must be atomic/reconcilable, never silently discard verified work. Round/token/runtime/cost caps are bounded and tied to the run contract; unknown provider usage is not zero. No unbounded transcript capture or storing model reasoning/secrets by default.

Relationships and anti-duplication

  • Completed #158/ADR 0004–0006: authority, candidate and evidence primitives.
  • Completed #176/ADR 0008: one worker/run per issue and L0/L1/L2; do not build another EPIC scheduler.
  • Open #147, #126, #154: SDK sessions, interaction and redacted usage. Consume their contracts where available, but no blanket requirement to finish #147 before the experimental CLI/Pi mode.
  • Existing #194, #199, loop_guard.py and workflow.stagnation_*: reuse and extend observability rather than duplicate generic tool-loop detection. A repeated task text alone is not conclusive stagnation.
  • Existing #266: sandbox-execution backend only, no hard prerequisite for planning the new feature. Every agent and verifier launch remains subject to whichever existing backend policy applies.
  • #23 owns persisted protocol number coordination. Do not claim a number or bypass migration review.
  • No hard dependency on Public Alpha #255 and do not add this EPIC to release gates. #184 decides top-level queue placement; creation here does not reorder #147/#266/#306.

Proposed waves (children to be linked below)

A — contract and evidence: A1 architecture ADR; A2 typed evidence/working-memory ledger; A3 reproducible benchmark and no-change baseline.
B — experimental execution: B1 fresh-context bounded handoffs; B2 verification-driven replan; B3 adaptive budget/routing and stop rules.
C — guarded options: C1 independent-perspective candidate branching; C2 ablations, operational rollout and go/no-go.

Success and release gates

  • ADR ratified with no new write authority, deterministic control, reviewed state/migration contract and explicit provider/session capabilities.
  • Typed bounded per-issue ledger proven durable across interrupted manager/worker invocations and issue/FIX candidate transitions; reconciliation does not mistake unverified notes as review evidence.
  • Controlled, reproducible single-agent vs single-agent+ledger vs adaptive manager/worker comparison on representative real repository issues, with the same model/provider, tool access, sandbox settings, validation and run-level resource limits wherever comparable. Record sample size, non-completions and statistical uncertainty, not cherry-picked successes.
  • Demonstrate objective verifier failure veto and successful replay/escape from repeated-failure approaches without bypassing Review, Git identity, evidence, CI or merge gates.
  • Measure success/verified-PR rate, review/fix cycles, human intervention/BLOCKED, cost per validated successful issue, tokens, wall time, provider/infra failures, safety violations, outcome drift. Document how unknown tokens/subscription price are handled.
  • No unbounded calls/storage or unexpected third-party model/API fallback; budget exhaustion yields existing safe stop/pause outcome with preserved candidate.
  • No degradation of existing default single-agent mode or provider execution; feature opt-in/experimental until predeclared decision thresholds are met.
  • Default remains off. C-stage gating may reject rollout if quality/cost/safety evidence is unfavorable.

Operational caveats

  • GVS5H's reported pass@1 gains concern competitive programming, not GitHub repository engineering. Research solution.py overwrite, prompt-text-hash workspace, raw reasoning transcripts, arbitrary subprocess verifier and model STATUS: done are not production templates.
  • The GVS5H MIT code can inspire patterns; no need to vendor or depend on its script. Keep the implementation native to AutoForge's Python controller and provider contracts.
  • Child issue text links are planning metadata unless native GitHub parent/blocked_by edges are created. Do not run autoforge epic run on this EPIC before the native graph is verified.

Child task ledger — eleven created issues

Planning ledger only: These are eleven separate open GitHub issues. The current connected GitHub tool supports issue body updates but not writing native sub-issue / blocked_by edges; the names and links below are not scheduler-readable edges.

Wave Task Issue Size Hard blockers Deliverable
A DEL-A1 #316 L — ADR: controller vs Manager/Worker/Verifier, safety and recovery
A DEL-A2 #317 L #316 Typed, bounded, trusted-storage Evidence Ledger
A DEL-A3 #318 L — Reproducible baseline and predeclared outcome/cost metrics
B0 DEL-B0 #324 L #316, #317 Native retrieval API; SQLite FTS5 default; optional LTSearch feasibility
B DEL-B1 #319 L #316, #317 Fresh-context provider-neutral Worker handoff
B DEL-B2 #320 L #317, #319 Verification-driven replan and stagnation escape
B DEL-B3 #321 M #316, #318, #319 Adaptive budgets, selective activation, usage
C0 DEL-C0 #325 L #317, #324 Review-gated Repository Knowledge promotion and lifecycle (post-MVP opt-in)
C DEL-C3 #326 L #316, #317, #325 Offline checkpointed consolidation; review-only derived memory proposals (optional post-MVP)
C DEL-C1 #322 L #320, #321 Separate independent-perspective candidates (opt-in; no auto parallel mutating workers)
C DEL-C2 #323 L #318, #320, #321 Ablations, security audit and go/no-go rollout

Topological order: #316 and #318 can start independently. #316 → #317 → #319 (basic fresh handoff), and #316 + #317 → #324 (default FTS5 and optional LTSearch feasibility); #317 + #324 → #325 (opt-in repository knowledge). #319 → #320, while #321 needs #318 and #319; then #323 and optional #322. Optional #325 → #326 offline consolidation; neither blocks original #323. C1 is not a hard blocker for C2: the independent-perspective treatment is included only if ready and explicitly approved.

MVP gate: #316–#321 pass acceptance and #318 documents baseline; final experimental rollout is governed by #323. #322 is exploratory and may be deferred/declined without blocking a safe feature-flagged Ledger+handoff MVP.

Scheduler-readiness action (not done)

Attach exactly #316–#326 (eleven distinct issues) as direct native GitHub sub-issues of #315 and create native blocked_by edges matching the table, by a capable GitHub CLI/API operator. Then read the graph back with the GraphReader/autoforge epic status contract: eleven distinct children, no cycles, and exactly the blockers listed above. Do not run autoforge epic run on #315 before this verification.

Portfolio routing

New independent post-alpha, opt-in research/implementation EPIC. No new hard dependency on #147 SDK, #266 sandbox, #306 cross-object GitHub actions, or Public Alpha #255. Coordinate with #184 portfolio roadmap and #23 before any persisted protocol migration.

Native Memory and pluggable retrieval decision (2026-10-24)

Adopted planning decision: AutoForge owns deterministic Memory state, approval, scope, evidence provenance, corrections, validity and freshness. SQLite FTS5 is the default, no-service retrieval backend; LTSearch is a feasibility candidate for optional hybrid retrieval, not Memory authority and not a mandatory dependency. Reuse/modify LTSearch code only after code-license/redistribution clearance, measured retrieval advantage and deployment/security review.

Three layers: (1) existing Controller-owned execution evidence; (2) candidate-bound intra-issue Ledger #317; (3) review-gated Repository Knowledge #325, never automatically global/cross-repository. Retrieval/search indexes are rebuildable projections, never an authority source. Withdrawals/corrections are immediately enforced by authoritative store read-back even while search index lags. No raw transcript capture, full prompt/reasoning retention, auto-install of Agent Skills, or Agent plugins; existing provider isolation stays intact.

New tasks: #324 native retrieval/FTS5 + optional LTSearch benchmark (hard deps #316/#317), and #325 approved Repository Knowledge promotion (hard deps #317/#324). The original GVS5H single-agent/handoff MVP is not blocked by either optional long-term memory workstream; basic #319 handoff uses curated #317 Ledger directly. #323 may add long-term-memory ablations if #325 finishes, but is not blocked by it.

References: TenseMem fact lifecycle/freshness, Beacon candidate review/promotion, LTSearch BM25/vector/RRF/index releases. Coordinate persisted protocol changes via #23. No new Public Alpha #255 gate.

Reference library and MemCurrent design follow-up (2026-10-25)

Track the following primary references when refining ADR #316 and evaluating Memory: These are architectural inspiration, not approved runtime dependencies, license clearance or evidence of performance on GitHub Issue → PR work.

Reference Primary resource What to examine AutoForge owner
GVS5H Paper PDF · Source Manager/Worker fresh contexts, curated ledger, feedback and no-progress guards; algorithm-problem results do not extrapolate to real repos #316, #319, #320, #318
TenseMem Repository · Architecture · TenseBench Typed fact state, corrections/supersession, temporal scope, stale-read flags, self-contamination guard and correction durability #316, #317, #318
Agent Beacon Repository · Review-gated memory · Session handoff Cross-harness lessons, human review/promotion, typed evidence/handoffs, privacy and limited context; no plugin installation or bulk transcript harvesting in AutoForge providers #317, #319, #325
LTSearch Repository · Architecture · Query router Tantivy BM25, LanceDB vector/RRF, index publication, query/filter/freshness, adapter feasibility, licensing and benchmark gate; SQLite FTS5 stays default #324
Databricks MemCurrent Design blog · Databricks Agent memory pipelines Episodic versus derived memory; online fixes + offline consolidation; compact memory index, two-stage search/fetch; checkpointed batch processing; ablation of end-to-end usefulness #316, #317, #318, #319, #323, #324, #325, #326
Accepted architecture additions
  • Online: current-Issue corrections and retractions take effect immediately in host Controller-owned #317 MemoryStore, even if #324 indexes are stale.
  • Offline: new #326 performs opt-in, checkpointed, bounded, replay/idempotency-safe consolidation of trusted-source sanitized summaries into proposed Repository Knowledge (duplicate/merge/split/conflict/staleness candidates). #325 is the sole approval/reject/retract authority; no auto-promotion by model score.
  • Retrieval: #324 implements limited search_memory then authorized fetch_memory, and a compact per-launch Memory Catalog; #319 gives only scoped, current and redacted projections. Use no globally installed Agent extensions/skills.
  • Evaluation: #318 predeclares memory-dependent vs memory-independent tasks and freshness/correction leaks; #323 optionally compares none/Issue Ledger/approved derived/combined retrieval, with full offline/online and reviewer cost. Do not infer AutoForge performance from Databricks internal benchmark percentages.
  • Scope: #326 is a post-MVP optional C task, not a new hard blocker for #319/#320/#321/#323 or Public Alpha #255. No change to controller-owned GitHub effects, sandbox boundary, single-agent defaults or #176 scheduler.
Graph write limitation

The ledger now identifies eleven child task issues. Prose links are not GitHub-native sub_issue or blocked_by edges. Attach #316–#326 (including #326) as native direct children of #315 and create exact blocker edges from this ledger before running AutoForge EPIC scheduler; read back and confirm no cycles/duplicates. No edge creation is asserted by this planning update.

Dominant language
Python
Stars
0
Forks
0
Avg merge
5h 51m
Merged PRs (30d)
66

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from iceboundrock/AutoForge

All issues in iceboundrock/AutoForge

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.