Generalize CI verification guidance in AGENTS.md to all yarn.lock patches

Open Beginner friendly
#2,925 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Domain
documentation

Research direction

Start in AGENTS.md by reading the “Leverage CI verification” and “Major Version Bumps in Patches” sections. Generalize the guidance to all patches/*.patch changes affecting yarn.lock, including the e2e-tests/ condition and the /publish, /smoketest, and /test recommendations. Done means the guidance is separated or referenced without duplication and can be checked on the next three relevant PR reviews.

Written by the indexing model from the issue text.

Description

documentation ready-to-code stale

What happened

PR #2763 modified workspaces/lightspeed/patches/0-cve-yarn-lock.patch with minor/patch-level dependency bumps (brace-expansion 1.1.11→1.1.16, 2.0.3→2.1.2, 5.0.5→5.0.7). The review agent (Run 1, Run 2) approved with no mention of verification steps (/publish, /smoketest, /test). The human author independently ran /publish and /test, which revealed a pre-existing E2E infrastructure failure. The PR was closed after 11 days and superseded by PR #2923.

What could go better

The AGENTS.md section "Leverage CI verification" — which recommends /publish, /smoketest, and /test as verification steps — is currently nested inside the "Major Version Bumps in Patches" section. Because PR #2763 had only minor/patch-level bumps, the review agent did not invoke this guidance. As a result, the review body contained no recommendation to verify the patch via CI commands.

For this specific PR, the experienced author knew the workflow and ran the commands independently. But for less experienced contributors, a review that says APPROVE with ready-for-merge but no mention of CI verification could lead to premature merge attempts — especially since E2E tests are not a required status check on release branches.

Confidence: Medium. The experienced author workflow worked here, but the gap becomes real for new contributors or when the ready-for-merge label issue (fullsend#1424) has not yet been resolved. This proposal is complementary to #2879 (merge gating for E2E), which solves the enforcement side while this addresses the guidance side.

Proposed change

In AGENTS.md, extract the CI verification guidance from the "Major Version Bumps in Patches" section into a new general-purpose section (e.g., "Reviewing Yarn Lock Patches") that applies to ALL patches/*.patch changes that modify yarn.lock content, regardless of whether a major version bump is detected.

The new section should instruct the review agent to:

  1. Check whether the affected workspace has an e2e-tests/ directory.
  2. Recommend /publish + /smoketest as minimum verification steps for any yarn.lock patch.
  3. If e2e-tests/ exists, additionally recommend /test e2e-ocp-helm for runtime verification.
  4. Note that passing /publish confirms the patch applies cleanly, while /smoketest confirms plugins load, and /test exercises runtime behavior.

The existing "Major Version Bumps in Patches" section would retain its specific criteria (detect major bump, check changelog, assess breaking APIs, flag for human verification) and reference the general CI verification guidance rather than duplicating it.

Validation criteria

On the next 3 PRs that modify patches/*.patch files with yarn.lock changes (any version bump level), the review agent's review body should include a recommendation to run /publish + /smoketest (minimum) and /test (if e2e-tests/ exists in the workspace). Verify by checking the review comments on those PRs for mentions of these PR commands.


Generated by retro agent from https://github.com/redhat-developer/rhdh-plugin-export-overlays/pull/2763

Dominant language
TypeScript
Stars
9
Forks
72
Avg merge
3d 9h
Merged PRs (30d)
133

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from redhat-developer/rhdh-plugin-export-overlays

All issues in redhat-developer/rhdh-plugin-export-overlays

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.