[quality] validate-members.mjs checks landscape revision and sourceUrl against the snapshot but not collectedAt, the one field the page renders
Maintainers usually reply within 1 day
A pull request for this has already been merged.
- #1312 by @mrbobbytables — merged
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 58/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- javascript, node.js
Research direction
The check to change is the provenance condition in scripts/validate-members.mjs around lines 420-428, and scripts/generate-members.mjs lines 302-306 show where collectedAt is written. Add collectedAt to that comparison against the snapshot's root field, then add a case to tests/validate-members.test.mjs. Done when a members.json with a stale or unparseable collectedAt yields a members.json.sources.landscape error, and the data-variant row in tests/e2e-data-fixture-validity.test.mjs is updated if #1269 has merged.
Written by the indexing model from the issue text.
Description
Finding
scripts/validate-members.mjs cross-checks the landscape provenance block in
data/members.json against the committed snapshot it was generated from, but
checks only two of its three fields (scripts/validate-members.mjs:420-428):
if (
data.sources?.landscape?.revision !== landscape.source?.revision ||
data.sources?.landscape?.sourceUrl !== landscape.source?.sourceUrl
) {
errors.push({
path: 'members.json.sources.landscape',
severity: 'error',
message: 'landscape provenance must match enduser-landscape.json',
});
}
collectedAt is the third field of that same block, written from the same object
in the same statement by the generator (scripts/generate-members.mjs:302-306):
generatedAt: snapshot.collectedAt,
sources: {
landscape: {
...snapshot.source,
collectedAt: snapshot.collectedAt,
},
},
and it is the one of the three that is rendered:
src/components/MemberDirectory/DirectoryFreshness.js:17 reads
landscape?.collectedAt through formatDate() to produce the "Directory
membership data last synced from ..." sentence.
So the two fields nobody sees are held to the snapshot, and the field the page
prints is held to nothing. Every sibling validator gates its own document-level
timestamp — validate-metrics.mjs (generatedAt must be ISO 8601),
validate-case-studies.mjs, validate-radar-reports.mjs,
validate-community-people.mjs (fetchedAt must be strict ISO 8601),
validate-community-groups.mjs (checkedAt must be ISO 8601),
validate-awards.mjs (verifiedAt must be a parseable date), and
validate-enduser-members.mjs:91 gates collectedAt on the upstream snapshot
itself. members.json is the one that does not.
The practical consequence is narrow but real: a members.json whose
collectedAt is stale relative to enduser-landscape.json — a partial
regeneration, a merge that took one side of the file, a hand-edit — passes
validate:members while DirectoryFreshness prints a sync date that is not the
one the data came from. An unparseable value passes too, and formatDate()'s
Number.isNaN guard silently drops the whole membership sentence.
Evidence and provenance
-
Repository revision
365e11f(main), clean checkout,npm ci, node v26.10.0. -
scripts/validate-members.mjswas run throughrunScriptInSandbox()
(tests/helpers-script-sandbox.mjs) over the realdata/tree with only
sources.landscape.collectedAtreplaced. The committed value matches the
snapshot exactly (2026-10-01T05:13:13.673Zin both files). All three mutations
produced zeromembers.json.sources.landscapeerrors and zero errors
mentioningcollectedAt:substituted value landscape/collectedAt errors "not a timestamp"none "1999-01-01T00:00:00.000Z"(valid, but not the snapshot's)none nullnone (The runs exit non-zero on unrelated
logo does not exist under static/
findings, which the sandbox produces for the unmodified tree as well because it
does not mirrorstatic/; those are identical across all four runs.) -
Independently corroborated by a fixture:
tests/e2e/fixtures/data-variant/members.json
setssources.landscape.collectedAtto"not a timestamp"precisely to degrade
that sentence, and the merged document adds no validator error — the only
overlay in the repository that deliberately breaks a document-level timestamp and
is not rejected for it. See the table in #1268. -
Unit.
npm run test:unit:coveragerun locally at this revision:
src files | 100.00 | 100.00 | 9015/9015 lines | 2713/2713 regions. Both the
validator andDirectoryFreshnessare fully covered, so this is a missing
rule, not a coverage gap, and is filed asregression-riskrather than
coverage-gap. -
End-to-end. Not claimed: Playwright's chromium could not be launched in this
environment (it needs ~20 OS libraries andinstall-depsrequires root), so no
browser percentage is asserted here. None is needed — the finding is read from
the validator's source, not from a measured shortfall.
Recommendation
One deliverable, one file, one mechanical edit: fold collectedAt into the
comparison that already exists, so all three fields of the block the generator
writes together are checked together.
-
Extend the existing condition at
scripts/validate-members.mjs:420-422:```diff if ( data.sources?.landscape?.revision !== landscape.source?.revision || - data.sources?.landscape?.sourceUrl !== landscape.source?.sourceUrl + data.sources?.landscape?.sourceUrl !== landscape.source?.sourceUrl || + data.sources?.landscape?.collectedAt !== landscape.collectedAt ) { ``` Note `collectedAt` sits on the snapshot root, not on `snapshot.source` — `generate-members.mjs` copies it from `snapshot.collectedAt` while spreading `...snapshot.source` for the other two. -
Add the matching case to
tests/validate-members.test.mjs, so the new arm is
covered and the rule is pinned rather than merely present. -
Update the
data-variant/members.jsonrow of the table in
tests/e2e-data-fixture-validity.test.mjs(added by #1269), whose entry
records today's empty delta and says in its comment that closing this gap is
what should make it gain an error. If #1269 has not merged, say so on it
instead; the two touch different files and can land in either order.
Deliberately not split: it is one condition, one test file and one table row.
Not opened as a PR by this agent. The first box changes
scripts/validate-members.mjs, which is production validator code; this lane is
restricted to testing changes and must not PR it. The exact replacement text is
given so applying it is mechanical. It needs a human or an ISSUES_PRS_MERGE
agent to land.
Priority
- Impact: low-medium (the only rendered field of the provenance block is the only
one not held to its source; the failure mode is a sync date the page states and
the data does not support, or a silently dropped sentence) - Effort: low
Filed by quality agent (hold-gated mode)
🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 5ccc4e5
— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88
- Dominant language
- JavaScript
- Stars
- 0
- Forks
- 3
- Avg merge
- 7h 15m
- Merged PRs (30d)
- 623
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from cncf/endusers
-
[sec-check] collect-metrics.mjs tallies upstream landscape category and maturity names on prototype-bearing objects, corrupting or dropping published countsPossibly taken @hivecommons-hive claimed this today. Openagent/sec-check security stage/040-covered-by-pr
Difficulty 1/5 Under an hour Newbie friendliness 25/100
Maintainers usually reply within 1 day
-
[sec-check] ip-address 10.7.2 carries ReDoS advisory GHSA-rxp7-6hxx-x8q5; patched 10.7.4 is a lockfile-only bumpPossibly taken @hivecommons-hive claimed this today. Openagent/sec-check security stage/040-covered-by-pr
Difficulty 1/5 Under an hour Newbie friendliness 10/100
Maintainers usually reply within 1 day
-
agent/strategist roadmap stage/045-needs-decision
Difficulty 5/5 Over a week Newbie friendliness 25/100
cncf/endusers#1822 · 1 comment ·
Maintainers usually reply within 1 day
-
[architect] Duplicated interactive-card hover/focus-visible CSS across CNCFProjectCard, MetricsDashboard, ReferenceArchitecturesPossibly taken @hivecommons-hive claimed this today. Openagent/architect architecture stage/040-covered-by-pr tech-debt
Difficulty 3/5 1-2 days Newbie friendliness 25/100
Maintainers usually reply within 1 day
-
[quality] Mutation testing discards the coverage of every script tests/helpers.mjs spawnsPossibly taken @hivecommons-hive claimed this today. Openagent/quality stage/040-covered-by-pr testing
Difficulty 4/5 3-5 days Newbie friendliness 25/100
cncf/endusers#1817 · 2 comments ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
lyswhut/lx-music-desktop#3026 ·
-
good first issue server
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
openstreetmap/iD#13054 ·
Maintainers usually reply within 1 day
-
community marketing priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
TMHSDigital/Free-Game-Dev-Assets#151 ·
Maintainers usually reply within 1 day
-
Read a vector index's quantization back on Neo4j 2026.10, which reports vector.quantization.typeOpen
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
liberation-data/drivine#231 ·
Maintainers usually reply within 1 day