Add AGENTS.md guidance for verifying multi-workspace E2E fix coverage

Open Beginner friendly
#3,696 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
1-3 hours
Newbie friendliness
90/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active

Research direction

Read AGENTS.md, especially the “E2E Nightly Fix Conventions” section around the “Allowed modifications” guidance near line 473. Add the proposed “Multi-workspace fix coverage” subsection, preserving its four verification steps. Done means the guidance explains how to enumerate affected workspaces, cross-reference changes, document exclusions, and verify completeness.

Written by the indexing model from the issue text.

Description

documentation ready-for-triage ready-to-code

What happened

PR #3354 attempted to fix issue #3347, which listed ~66 test failures across 21+ workspaces with keycloak auth. The code agent modified 14 workspaces (18 files) but missed 3 keycloak-auth workspaces: argocd, intelligent-assistant, and app-defaults. The issue body listed all initially-affected workspaces, and triage recurrence comments on Aug 22–25 added argocd and intelligent-assistant to the affected list. The code agent did not cross-reference its changes against the issue’s affected-workspaces list, resulting in incomplete coverage that would have left those workspaces broken even if the OCI references had been correct.

What could go better

The code agent should systematically verify that its fix covers all components listed as affected in the issue. AGENTS.md has extensive E2E guidance (auto-generation behavior, config file creation, best practices) but no instruction to cross-check fix scope against the issue’s stated impact. When an issue lists 21+ affected workspaces, the code agent should enumerate all of them, verify each is addressed (by the fix or by the framework), and flag any discrepancies. This is especially important for cross-cutting E2E issues where the affected list evolves through triage recurrence comments. Confidence: high — the issue body and triage comments contained the complete affected workspace list, and the 3 missed workspaces (argocd, intelligent-assistant, app-defaults) were all identifiable from available information.

Proposed change

Add a subsection to AGENTS.md under ‘E2E Nightly Fix Conventions’ (after ‘Allowed modifications’, around line 473):

### Multi-workspace fix coverage

When an E2E failure issue lists specific affected workspaces or projects:

1. **Enumerate** all workspaces listed in the issue body and any triage recurrence comments as affected
2. **Cross-reference** each affected workspace against your proposed changes
3. **Document exclusions** — for any affected workspace not covered by the fix, add a comment in your commit message or PR body explaining why (e.g., “relies on auto-generation from metadata; the framework-level fix in e2e-test-utils handles this” or “uses guest auth, not affected by OIDC issue”)
4. **Verify completeness** before committing — `grep -r` for the pattern you’re adding/modifying across all workspace e2e-tests directories to confirm you haven’t missed any existing configs that need the same change

Validation criteria

The next code agent run that fixes an E2E failure affecting multiple workspaces either (a) covers all workspaces listed in the issue or (b) explicitly documents in its commit message or PR body why specific workspaces were excluded. Measurable over the next 3 multi-workspace E2E fix PRs.


Generated by retro agent from https://github.com/redhat-developer/rhdh-plugin-export-overlays/pull/3354

Dominant language
TypeScript
Stars
9
Forks
72
Avg merge
3d 9h
Merged PRs (30d)
133

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from redhat-developer/rhdh-plugin-export-overlays

All issues in redhat-developer/rhdh-plugin-export-overlays

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.