Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

feat(review): multi-call prompt decomposition roadmap (v0.7–v0.9) — dedicated prompts per pipeline stage, non-agentic

Open
  • Is something wrong?
#669 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
8/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Quiet
Tech stack
java
Domain
ai

Research direction

This is an umbrella roadmap rather than a single change, so there is no one file to start from. Read PrReviewPrompts.SYSTEM, FindingPipeline.runMultiCall and DiffBudgetPlanner.plan to see how the review calls are built and budgeted, and check PrReviewPromptsContentTest for the guidance that must keep passing. Pick one linked child issue, such as #664 or #665, rather than this roadmap.

Written by the indexing model from the issue text.

Description

enhancement java
Problem Statement

Post-v0.6.0 direction: move review efficiency forward by decomposing the review flow into more, smaller, dedicated-prompt LLM calls — deterministic orchestration, not agentic — so each call has a predictable size and a single concern.

The orchestration is already multi-call (FindingPipeline.runMultiCall: N batches + per-batch verifier + summary, capped by REVIEW_MAX_AI_CALLS), and FindingVerificationService is already a dedicated second-stage call. What is still "single-shot" is the review call itself: PrReviewPrompts.SYSTEM (~600 lines) makes one call handle 10 review dimensions, severity/confidence calibration, previous-findings reconciliation, suggestions, and the full summary object. Decomposing that mega-prompt — and bounding follow-up rounds by push size instead of PR size — is this roadmap.

Phases

v0.7 — slim the review call (priority)

  • #664 — make the summary call universal; review call returns findings + statuses only
  • #665 — per-batch dimension routing; system prompt assembled from only the relevant dimension blocks

v0.7 — verification correctness

The verification stage lacks the evidence to rule correctly; each ticket is committed to a single approach and they compose rather than overlap:

  • #650 — deterministic cited-location resolution: resolve each finding's path:line (quote-search-first) against the real file and hand the snippet to the verifier as established material. No model call.
  • #475 — self-carrying evidence: context-grounded findings quote the measurement/rule they rest on; the verifier judges the quoted grounding. No new verifier slots, stable inputs as context dimensions grow.

v0.8 — smarter context selection

  • #666 — file triage stage: classification drives packing order, prompt selection, and feeds #43 (risk-based model routing)
  • #55 — adjacent-file context: deterministic reference extraction + one small selection call + budget-share-bounded injection, fed to reviewer and verifier

v0.9 — predictable follow-up rounds

  • #667 — incremental re-review on synchronize + dedicated finding-resolution check call; prompt size bounded by push size

Opt-in / cross-cutting

  • #668 — targeted per-finding verification context (the composition point where #650's and #55's material attaches to exactly the finding it belongs to)
  • #670 — structured outputs (response_format: json_schema) behind a per-model capability flag

Agentic directions (#63 parallel specialists, #57 agentic fix) are post-1.0, opt-in, not currently planned — this roadmap is the non-agentic path; #665 delivers the "dedicated prompt per concern" benefit without agents or tool-calling.

Related open issues, sequenced against this roadmap

Fix before/while landing the decomposition (they live in code these phases restructure):

  • #623 — verification silently not running on ~1/3 of reviews; must be fixed before more stages are modeled on the verifier pattern (#667's resolution check)
  • #622 — context-length rejection retried 5× at full price; more calls = more exposure
  • #636 — pr_purpose/MarkdownSafe fix must land on the summary-call path, since #664 moves pr_purpose there
  • #655 / #659 — disclosure/coverage-clause fixes should land before #664/#666 restructure that logic

Made more urgent by this roadmap:

  • #28 — cached/reasoning-token cost accounting: the cost model here leans on prompt caching absorbing the fan-out (byte-stable prefixes below); two-bucket pricing will overstate the pipeline's cost and hide whether the decomposition pays off
  • #239 — per-model tokenizer: estimation error compounds per call once many smaller calls each carry a budget share
  • #604 — random fence non-reproducibility also works against stable-prefix caching

Fed by this roadmap:

  • #43 — risk-based model routing consumes #666's classifier
  • #59 / #34 / #60 — deterministic context inputs (CI failures, linters/SARIF, secrets/IaC scan); each should follow the budgeted fenced-section pattern, and their signals can feed #666's risk tiers
Constraints every stage must respect
  • Every AI call goes through ReviewTokenLedger.ensureCallAllowed with usage recorded (streaming via OtelObservabilityListener, blocking via Result.tokenUsage()), or REVIEW_MAX_TOKENS_PER_REVIEW silently breaks.
  • DiffBudgetPlanner.plan reserves calls out of REVIEW_MAX_AI_CALLS (currently exactly one, for the summary) — each new stage changes that accounting explicitly.
  • Budget overhead estimation concatenates the actual prompt constants (DiffBudgetPlanner.plan, FindingPipeline.clampOverview / budgetedFindingsJson) — every prompt split updates those sites or the budgets become fiction.
  • PrReviewPromptsContentTest pins the guidance that past issues fought for; the eval corpus (-Peval) is the recall regression gate. Prompt restructuring lands only with both green, and the risky splits ship behind config gates with the current behavior as fallback.
  • Keep system-prompt prefixes byte-stable within a review so OpenAI-compatible prompt caching keeps paying for the multi-call fan-out.
Priority

Important

Dominant language
Java
Stars
35
Forks
7
Avg merge
9h 23m
Merged PRs (30d)
154

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from devops-thiago/ThrillhouseBot

All issues in devops-thiago/ThrillhouseBot

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.