issueAsk rail demotes a reviewer APPROVE for an ask that is present in the diff
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 42/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- go, kubernetes
- Domain
- ai-infra-agents, backend
Research direction
Start by reading enforceReviewerIssueAsk in pkg/foreman/agent/executor_native.go and the related guard in internal/foreman/controller/workload_iteration.go. Trace how reviewer verdicts and issue ask verification are combined, then inspect relevant tests or add a regression case for an explicit APPROVE with an unverified ask. Done means the rail no longer turns that approval into a rejection without a basis.
Written by the indexing model from the issue text.
Description
Bug Description
enforceReviewerIssueAsk (pkg/foreman/agent/executor_native.go) rewrote a reviewer's APPROVE to NO-GO because it could not verify the stated issue ask — while that ask was implemented in the diff under review, at four separate call sites.
The rail's own hazard is already documented at internal/foreman/controller/workload_iteration.go:175:
That is a statement about verification confidence, not about the change... re-running the coder cannot make an unverifiable issueAsk verify, so the loop has no terminating condition.
The controller guards against that by refusing to open a fix iteration (workload_iteration.go:220). But nothing prevents the demotion itself, so a correct, approved change is turned into a rejection, no PR is opened, and any external retry loop spends its whole budget re-running work that was already right.
Steps to Reproduce
- File an issue whose body contains a numbered "Suggested shape" list, so one list item reads as a discrete, checkable ask.
- Let a coder implement it and return
GO. - Let the reviewer review it and reach
APPROVE.
Observed on misospace/dispatch#940, an issue whose body had a four-point suggestion list.
Expected Behavior
A rail that cannot verify an ask should not convert an explicit reviewer APPROVE into a rejection. Either it downgrades to a warning and lets the reviewer's own verdict stand, or the demotion is gated on the reviewer having actually had the means to verify.
Actual Behavior
Audit record for the review task:
"verdict": "NO-GO",
"reviewer": { "outcome": "APPROVE" },
"issueAsk": { "verified": false,
"claimed": "Record `headSha` at enqueue so a comparison is possible at all." },
"turnCount": 18,
"elapsedSec": 184.8
The claimed string is item 1 of the issue's "Suggested shape" list. The coder's branch implements exactly that:
02bab1e fix(pr-fix-queue): refuse FIXED on no-progress and reopen stale tombstones (#940)
8 files changed, 450 insertions(+), 20 deletions(-)
with headSha: prHeadSha written at enqueue in four call sites. So the rail marked unverified a change sitting in the diff it was reviewing.
Downstream: no PR (the reviewer opens one only on GO), three identical coder attempts at turnCount 158 each, and the issue parked for a human. The 450 lines are still on the branch, unmerged.
Possibly relevant
The reviewer ran 18 turns in 185s. I could not confirm what it did — transcriptRef points at foreman-transcript-wl-misospace-dispatch-940-review-940-0, which does not exist in the namespace, so the transcript for this review is unavailable.
A separate review in the same cluster stated in its own output:
The native tool-calling loop issued no tool calls; review was performed against the provided corpus only.
If a reviewer reaches its verdict without issuing tool calls, it has no means to verify an ask against the diff, and verified: false would be the expected outcome for every ask rather than a signal about this change. That is a hypothesis, not something I can demonstrate for this run.
Environment
- LLMKube operator and coder image
0.9.24 - Agent
reviewer,mode: InProcess,maxTurns: 80 - Model: gemma-4-12b-it-qat via a LiteLLM proxy
contextStrategy: session,contextWindowTokens: 170000
Note
Filed from operating a Foreman deployment; happy to gather more from this cluster if a specific field or code path would help. The missing review transcript may itself be worth a look — an audit record whose transcriptRef resolves to nothing removes the only evidence of how a verdict was reached.
- Dominant language
- Go
- Stars
- 232
- Forks
- 37
- Avg merge
- 21h 42m
- Merged PRs (30d)
- 244
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from defilantech/LLMKube
-
component/metal-agent documentation
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
defilantech/LLMKube#2581 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
defilantech/LLMKube#2578 ·
Maintainers usually reply within 1 day
-
documentation helm priority/low size/small
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
defilantech/LLMKube#2420 ·
Maintainers usually reply within 1 day
-
component/metal-agent enhancement priority/low size/medium
Difficulty 4/5 3-5 days Newbie friendliness 48/100
defilantech/LLMKube#2597 ·
Maintainers usually reply within 1 day
-
[BUG] Docs and comments describe InsufficientMemory only as a start refusal, but a serving engine can now carry itPossibly taken @defilan-agent claimed this today. Openbug component/metal-agent documentation priority/low size/small
Difficulty 2/5 1-3 hours Newbie friendliness 25/100
defilantech/LLMKube#2596 ·
Maintainers usually reply within 1 day
All issues in defilantech/LLMKube
Similar issues
-
☢️ Bug v3
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
area/import-export area/projects concern/ux
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
go-vikunja/vikunja#4455 · 1 comment ·
Maintainers usually reply within 2 days
-
Dependabot PRs no longer merge automaticallyPossibly taken @dwisiswant0 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
projectdiscovery/urlfinder#160 ·
Maintainers usually reply within 1 day
-
database
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
chnm/legal-modernism#450 ·
Maintainers usually reply within 1 day
-
tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day