fix(review): resume requested reviews when checks become green
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
Research direction
The issue involves extending the github-pr-reviewer automation in the OpenHands/extensions repository. Start by examining the existing automation code, likely in a directory like skills/github-pr-reviewer. Understand the current triggers (event-based) and the changes from PR #651. The new cron trigger must scan open non-draft PRs with outstanding all-hands-bot review requests. Check how current-head checks are evaluated and the waiting/blocked disposition. Tests need to cover pending-to-green, failed-to-green on same SHA, changed head, and duplicate notifications. 'Done' means the automation works with a cron trigger, meets all acceptance criteria, and has passing tests.
Written by the indexing model from the issue text.
Description
Problem
The live OSS PR reviewer on oss-agent-canvas (f36aba99-070d-40a9-9f77-627f9113347a) has only a GitHub review_requested / bot review.submitted event trigger. Extensions #651 correctly stops before launching an agent while current-head checks are pending or failing, but its message promises a scheduled retry that this deployment does not have. A request arriving during CI can therefore be left unreviewed after CI turns green unless a human removes and re-requests all-hands-bot. This undermines automatic coverage of non-draft PRs.
Desired behavior
Keep the cheap current-head eligibility gate from #651, then automatically revisit a pending requested review when its current-head checks reach a terminal green state. Reuse the existing reviewer automation and conversation-dispatch machinery. Choose the smallest reliable trigger mechanism supported by Canvas Automations; do not launch an agent or consume a slot while checks are pending or failing.
Acceptance criteria
- A native
all-hands-botreview request made before CI finishes results in one review after the exact requested head's checks are green, without a second manual review request. - A failing current-head check still prevents review, and a later fix on the same head (such as editing a PR description) or a new head can be reconsidered without a stale failure blocking it.
- Repeated check-completion events or scans do not create duplicate conversations or reviews; an already completed review is not restarted.
- The deployed OSS reviewer configuration uses the retry mechanism on all four repos, and the user-facing waiting comment names the actual retry condition instead of promising a nonexistent scan.
- Tests cover pending-to-green, failed-to-green on the same SHA, changed head, and duplicate notifications. Live Canvas evidence shows a review requested while a check is pending and completed after it becomes green.
Related: #644 and #651. The currently live reviewer remains on the tested #652 bundle; do not merge or deploy #651 until its repeated-check-run policy and this retry path are resolved.
OpenHands AI triage
The following comments and acceptance criteria were added by the OpenHands AI agent.
Triage
The maintainer resolved the earlier design questions, so the scope is now bounded and does not need a further decision. The reusable reviewer behavior belongs in this registry, reusing the existing skills/github-pr-reviewer worker and conversation-dispatch machinery, and stacking on #651 (which must merge first). The OSS deployment gets its retry path by adding a five-minute cron trigger to the same reviewer automation record, scanning open non-draft PRs that hold an outstanding all-hands-bot review request, while the explicit-request event path is retained for other deployments.
Non-goals: no new GitHub webhook event types or check_run parsing (Canvas Automations does not parse that payload today, and adding it belongs to OpenHands/automation); no second automation record for the OSS deployment; no deployment-specific code in OpenHands/extensions. Updating the live oss-agent-canvas automation to the cron trigger on all four repos and syncing OpenHands/oss-agent-canvas is validation and deployment work performed after the PR is live-tested, not part of this change. Neither PR is merged or deployed automatically.
Acceptance Criteria
- The
github-pr-reviewerautomation can run on a recurring cron trigger and, on each run, considers open non-draft PRs that carry an outstandingall-hands-botreview request, in addition to the existing trigger-label scan and the explicit-request event path. - A native
all-hands-botreview request made before CI finishes results in exactly one review after the requested head's checks become green, with no second manual review request. - While the exact head has pending or failing checks, the run creates no review conversation and consumes no agent slot; it exits with a clear waiting/blocked disposition.
- A completed failing or cancelled check on the exact head still prevents review, and the head is reconsidered once checks pass.
- Current check state is determined from the latest run per logical check on the exact head, grouped by check name together with the reporting app identity and ordered deterministically by run start time and ID; a later successful run replaces an earlier failure on the same SHA, and a later pending rerun makes the head wait. Unrelated apps sharing a display name do not mask one another.
- Checks attributed to any other head do not block the current head, and a changed head is evaluated on its own checks.
- Repeated cron scans or duplicate notifications do not create duplicate conversations or reviews, and an already completed exact-head review is not restarted (existing keyed subject/delivery and completed-review dedupe reused).
- The user-facing waiting comment names the actual retry condition (the scheduled scan, or a new review request) rather than promising a mechanism the deployment does not have, and a moved head updates or replaces the previous gate comment instead of stacking a duplicate.
- Existing review verdict, maintainer-handoff, profile, and secret-handling behavior is unchanged, and no new secrets or elevated permissions are introduced.
- Automated tests cover pending-to-green, failed-to-green on the same SHA (latest-run-per-check), changed head, and duplicate notifications/scans.
- Live Canvas evidence demonstrates a review requested while a check is pending and completed after the check becomes green, and the documented
SKILL.md/README.mdretry contract matches the deployed cron behavior.
- Dominant language
- Python
- Stars
- 184
- Forks
- 137
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 46
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenHands/extensions
-
priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
OpenHands/extensions#975 ·
Maintainers usually reply within 1 day
-
priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
OpenHands/extensions#973 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
OpenHands/extensions#970 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
OpenHands/extensions#969 ·
Maintainers usually reply within 1 day
-
Add a `maven` skill: agents install Debian's `maven` package, which can't load plugins built for Java 17+Possibly taken @sumitvairagar claimed this 3 days ago. Openpriority:medium ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
OpenHands/extensions#963 ·
Maintainers usually reply within 1 day
All issues in OpenHands/extensions
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
huggingface/datasets#9128 ·
Maintainers usually reply within 1 day
-
Function/dict state handlers can't supply provider-state injected values (`generate.provider_state`)Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
pact-foundation/pact-python#1789 ·
Maintainers usually reply within 9 days
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
juandresrodca/AdminForge#66 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
backend/local comp/tools P2 tool/terminal type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NousResearch/hermes-agent#158788 ·
Maintainers usually reply within 1 day