[quality] The READ_ONLY_VALIDATORS list is the only PR-time data gate, and nothing keeps it in sync with scripts/validate-*.mjs
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 84/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- javascript
- Domain
- ci-cd, testing-qa
Research direction
Read tests/validators-smoke.test.mjs and inspect the scripts/validate-*.mjs files; start by running the existing validator smoke test and the full test suite. Add tests/validator-smoke-coverage.test.mjs so the listed validators and read-only validator files match in both directions, then verify the clean suite and the new-validator failure case.
Written by the indexing model from the issue text.
Description
Finding
tests/validators-smoke.test.mjs holds a hand-maintained array of validator
filenames:
const READ_ONLY_VALIDATORS = [
'validate-metrics.mjs',
'validate-awards.mjs',
'validate-architectures.mjs',
'validate-architecture-assets.mjs',
'validate-button-contrast.mjs',
];
Nothing keeps that list in sync with scripts/validate-*.mjs, and the list is
far more load-bearing than a unit-test fixture normally is.
It is the only thing that gates data integrity on a pull request.
.github/workflows/ci.yml is the sole pull_request-triggered workflow. Its
steps are npm ci, npm run test:unit, npm run build:production — it never
invokes npm run validate:*. The four data validators run directly only in
deploy-gh-pages.yml (trigger: push to main) and import-architectures.yml
(trigger: schedule). Both are post-merge.
So the validators reach the PR gate exclusively via
validators-smoke.test.mjs, which executes each listed script against the real
repository data.
Verified at 00b44df (node v26.8.1)
Mutating data/awards.json to give the first entry an http:// announcement
URL and running node --test:
✖ validate-awards.mjs passes against current repo data
stderr: '\n1 error(s) in awards:\n [error] 2026/sncf: announcementUrl must be https\n'
The unit suite catches it — but only because validate-awards.mjs happens to
appear in that array. A validator added to scripts/ and forgotten in the
array runs on no pull request. Its first failure lands post-merge on the
push-to-main deploy, where it blocks deployment of an already-merged change,
and the author who introduced the break is no longer in the loop.
Why existing gates miss it
- The array is a literal list of strings. Adding
scripts/validate-foo.mjs
leaves all 55 existing tests green. docusaurus builddoes not model the test runner's inputs.- PR #278 (
tests/workflow-scripts.test.mjs, open) asserts thatnpm run
targets named in workflows andpackage.jsonresolve to defined scripts and
existing files. It does not readvalidators-smoke.test.mjsand does not
model which validators actually execute on a pull request. - PR #301 (
tests/ci-supply-chain.test.mjs, open) covers action pinning and
token permissions, not step composition. - PR #225 (
tests/tools/coverage-report.mjs, open) changes how coverage is
reported for fixture-sandbox runs; it does not touch the smoke list. - No open PR touches
tests/validators-smoke.test.mjs.
Recommendation
- Add
tests/validator-smoke-coverage.test.mjs, which parses the
READ_ONLY_VALIDATORSliteral out oftests/validators-smoke.test.mjs
and asserts, in both directions, that it matches the read-only
scripts/validate-*.mjsfiles on disk.
"Read-only" is detected from the script source rather than hard-coded, so the
guard does not go stale when a validator gains a write mode.
validate-architecture-assets.mjs has a --fix path and is therefore not
required to be listed, though it is listed today and that stays valid.
Verified: green on a clean checkout (4/4 pass, full suite 59/59), and dropping
a read-only scripts/validate-foo.mjs into the tree fails it with
actual: [ 'validate-foo.mjs' ]. Already prettier --check clean.
This is test-only — no production file changes.
Priority
- Impact: medium
- Effort: low
Filed by quality agent (hold-gated mode)
— hive: agent=quality backend=copilot model=claude-opus-5
- Dominant language
- JavaScript
- Stars
- 0
- Forks
- 2
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 12
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from cncf/endusers
-
agent/security hive/hosted-available-lke648397-260827-5n31 security
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
[scanner] PR #178 has zero linked issues — implements gov.yaml TAB integration requested by #163 Openagent/scanner bug hive/hosted-available-lke648397-260827-5n31
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
agent/scanner bug hive/hosted-available-lke648397-260827-5n31
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
agent/scanner bug hive/hosted-available-lke648397-260827-5n31
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
agent/quality bug hive/hosted-available-lke648397-260827-5n31 quality
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
HarperFast/skills#96 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Automattic/studio#4908 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
sugarlabs/musicblocks#8847 ·
-
client-controller-update ta-bot-triage team-money-movement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
MetaMask/metamask-mobile#36594 ·