Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[quality] validate-members.mjs checks landscape revision and sourceUrl against the snapshot but not collectedAt, the one field the page renders

Closed
#1,270 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

A pull request for this has already been merged.

  • #1312 by @mrbobbytables — merged

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
58/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
javascript, node.js
Domain
data, tooling

Research direction

The check to change is the provenance condition in scripts/validate-members.mjs around lines 420-428, and scripts/generate-members.mjs lines 302-306 show where collectedAt is written. Add collectedAt to that comparison against the snapshot's root field, then add a case to tests/validate-members.test.mjs. Done when a members.json with a stale or unparseable collectedAt yields a members.json.sources.landscape error, and the data-variant row in tests/e2e-data-fixture-validity.test.mjs is updated if #1269 has merged.

Written by the indexing model from the issue text.

Description

agent/quality hive/hosted-available-lke648397-260827-5n31 quality testing

Finding

scripts/validate-members.mjs cross-checks the landscape provenance block in
data/members.json against the committed snapshot it was generated from, but
checks only two of its three fields (scripts/validate-members.mjs:420-428):

if (
  data.sources?.landscape?.revision !== landscape.source?.revision ||
  data.sources?.landscape?.sourceUrl !== landscape.source?.sourceUrl
) {
  errors.push({
    path: 'members.json.sources.landscape',
    severity: 'error',
    message: 'landscape provenance must match enduser-landscape.json',
  });
}

collectedAt is the third field of that same block, written from the same object
in the same statement by the generator (scripts/generate-members.mjs:302-306):

generatedAt: snapshot.collectedAt,
sources: {
  landscape: {
    ...snapshot.source,
    collectedAt: snapshot.collectedAt,
  },
},

and it is the one of the three that is rendered:
src/components/MemberDirectory/DirectoryFreshness.js:17 reads
landscape?.collectedAt through formatDate() to produce the "Directory
membership data last synced from ..." sentence.

So the two fields nobody sees are held to the snapshot, and the field the page
prints is held to nothing. Every sibling validator gates its own document-level
timestamp — validate-metrics.mjs (generatedAt must be ISO 8601),
validate-case-studies.mjs, validate-radar-reports.mjs,
validate-community-people.mjs (fetchedAt must be strict ISO 8601),
validate-community-groups.mjs (checkedAt must be ISO 8601),
validate-awards.mjs (verifiedAt must be a parseable date), and
validate-enduser-members.mjs:91 gates collectedAt on the upstream snapshot
itself. members.json is the one that does not.

The practical consequence is narrow but real: a members.json whose
collectedAt is stale relative to enduser-landscape.json — a partial
regeneration, a merge that took one side of the file, a hand-edit — passes
validate:members while DirectoryFreshness prints a sync date that is not the
one the data came from. An unparseable value passes too, and formatDate()'s
Number.isNaN guard silently drops the whole membership sentence.

Evidence and provenance

  • Repository revision 365e11f (main), clean checkout, npm ci, node v26.10.0.

  • scripts/validate-members.mjs was run through runScriptInSandbox()
    (tests/helpers-script-sandbox.mjs) over the real data/ tree with only
    sources.landscape.collectedAt replaced. The committed value matches the
    snapshot exactly (2026-10-01T05:13:13.673Z in both files). All three mutations
    produced zero members.json.sources.landscape errors and zero errors
    mentioning collectedAt:

    substituted value landscape/collectedAt errors
    "not a timestamp" none
    "1999-01-01T00:00:00.000Z" (valid, but not the snapshot's) none
    null none

    (The runs exit non-zero on unrelated logo does not exist under static/
    findings, which the sandbox produces for the unmodified tree as well because it
    does not mirror static/; those are identical across all four runs.)

  • Independently corroborated by a fixture: tests/e2e/fixtures/data-variant/members.json
    sets sources.landscape.collectedAt to "not a timestamp" precisely to degrade
    that sentence, and the merged document adds no validator error — the only
    overlay in the repository that deliberately breaks a document-level timestamp and
    is not rejected for it. See the table in #1268.

  • Unit. npm run test:unit:coverage run locally at this revision:
    src files | 100.00 | 100.00 | 9015/9015 lines | 2713/2713 regions. Both the
    validator and DirectoryFreshness are fully covered, so this is a missing
    rule
    , not a coverage gap, and is filed as regression-risk rather than
    coverage-gap.

  • End-to-end. Not claimed: Playwright's chromium could not be launched in this
    environment (it needs ~20 OS libraries and install-deps requires root), so no
    browser percentage is asserted here. None is needed — the finding is read from
    the validator's source, not from a measured shortfall.

Recommendation

One deliverable, one file, one mechanical edit: fold collectedAt into the
comparison that already exists, so all three fields of the block the generator
writes together are checked together.

  • Extend the existing condition at scripts/validate-members.mjs:420-422:

    ```diff
       if (
         data.sources?.landscape?.revision !== landscape.source?.revision ||
    -    data.sources?.landscape?.sourceUrl !== landscape.source?.sourceUrl
    +    data.sources?.landscape?.sourceUrl !== landscape.source?.sourceUrl ||
    +    data.sources?.landscape?.collectedAt !== landscape.collectedAt
       ) {
    ```
    
    Note `collectedAt` sits on the snapshot root, not on `snapshot.source` —
    `generate-members.mjs` copies it from `snapshot.collectedAt` while spreading
    `...snapshot.source` for the other two.
    
  • Add the matching case to tests/validate-members.test.mjs, so the new arm is
    covered and the rule is pinned rather than merely present.

  • Update the data-variant/members.json row of the table in
    tests/e2e-data-fixture-validity.test.mjs (added by #1269), whose entry
    records today's empty delta and says in its comment that closing this gap is
    what should make it gain an error. If #1269 has not merged, say so on it
    instead; the two touch different files and can land in either order.

Deliberately not split: it is one condition, one test file and one table row.

Not opened as a PR by this agent. The first box changes
scripts/validate-members.mjs, which is production validator code; this lane is
restricted to testing changes and must not PR it. The exact replacement text is
given so applying it is mechanical. It needs a human or an ISSUES_PRS_MERGE
agent to land.

Priority

  • Impact: low-medium (the only rendered field of the provenance block is the only
    one not held to its source; the failure mode is a sync date the page states and
    the data does not support, or a silently dropped sentence)
  • Effort: low

Filed by quality agent (hold-gated mode)


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 5ccc4e5

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Dominant language
JavaScript
Stars
0
Forks
3
Avg merge
7h 15m
Merged PRs (30d)
623

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cncf/endusers

All issues in cncf/endusers

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.