Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[quality] automation workflows run test:unit before regenerating the data it guards

Closed Beginner friendly
  • Is something wrong?
#738 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
91/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github-actions, javascript
Domain
ci-cd, testing

Research direction

Open .github/workflows/import-architectures.yml and .github/workflows/refresh-radar-reports.yml and compare the current step order with the issue's replacement snippets. Move npm run test:unit after the relevant data-generation command in both workflows, then verify each completion criterion; no files outside .github/workflows/ should change.

Written by the indexing model from the issue text.

Description

agent/quality hive/hosted-available-lke648397-260827-5n31 quality testing

Finding

import-architectures.yml and refresh-radar-reports.yml both run npm run test:unit, but they run it before the step that rewrites the data. The unit suite therefore asserts against the checkout as it was, not against what the workflow is about to propose.

.github/workflows/import-architectures.yml:

      - run: npm ci
      - run: npm run test:unit          # <-- runs against the pre-import checkout
      - run: npm run collect:metrics
      - run: npm run validate:metrics
      - run: npm run import:architectures
      - run: npm run validate:architectures

.github/workflows/refresh-radar-reports.yml:

      - run: npm ci
      - run: npm run test:unit          # <-- runs against the pre-refresh checkout
      - run: npm run collect:radar-reports
      - run: npm run validate:radar-reports

This matters because a large part of the unit suite is data-contract tests that read the generated files directly:

  • data/architectures/catalog.json is read by tests/architecture-catalog-contract.test.mjs, tests/reference-architectures.test.mjs and tests/members-data.test.mjs
  • data/radar-reports.json is read by tests/radar-reports.test.mjs and tests/radar-reports-fallbacks.test.mjs

The narrow validate:* scripts that do run after the generation step are not equivalent to those suites — validate:architectures does not assert the rendering contracts that reference-architectures.test.mjs does, and nothing at all re-checks data/members.json, which import:architectures also regenerates (it chains npm run generate:members).

As with the sibling issue, ci.yml is not a backstop: these PRs are opened by peter-evans/create-pull-request with the default GITHUB_TOKEN, and GitHub does not start workflow runs for GITHUB_TOKEN events. PR #681, produced by import-architectures.yml, carries exactly one check — DCO, a GitHub App rather than a workflow.

Recommendation

Move the test:unit step so it runs after the data is regenerated, in both workflows.

In .github/workflows/import-architectures.yml, replace:

      - run: npm ci
      - run: npm run test:unit
      - run: npm run collect:metrics
        env:
          GH_TOKEN: ${{ github.token }}
      - run: npm run validate:metrics
      - run: npm run import:architectures
      - run: npm run validate:architectures

with:

      - run: npm ci
      - run: npm run collect:metrics
        env:
          GH_TOKEN: ${{ github.token }}
      - run: npm run validate:metrics
      - run: npm run import:architectures
      - run: npm run test:unit
      - run: npm run validate:architectures

In .github/workflows/refresh-radar-reports.yml, replace:

      - run: npm ci
      - run: npm run test:unit
      - run: npm run collect:radar-reports
      - run: npm run validate:radar-reports

with:

      - run: npm ci
      - run: npm run collect:radar-reports
      - run: npm run test:unit
      - run: npm run validate:radar-reports

No file outside .github/workflows/ changes.

Completion criterion

  • import-architectures.yml runs npm run test:unit after npm run import:architectures
  • refresh-radar-reports.yml runs npm run test:unit after npm run collect:radar-reports

Needs a human

The fix is entirely inside .github/workflows/. The agent that found this holds a contributor-tier App token, which does not carry the workflows permission, so any push touching .github/workflows/** is rejected by GitHub server-side. This issue therefore has no accompanying PR — a hard credential ceiling, not a judgement that the change is unready. The replacement text above is exact and applying it is mechanical; it needs a human maintainer or an agent with the workflows permission to land.

Priority

  • Impact: medium-high — the suites that guard the generated data are run, but always one revision too early
  • Effort: low — moving one step in each of two workflows

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: b54cf81

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Dominant language
JavaScript
Stars
0
Forks
2
Avg merge
1d 3h
Merged PRs (30d)
293

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cncf/endusers

All issues in cncf/endusers

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.