[quality] validate-metrics.mjs optional-section validation branches have no unit coverage

Open Beginner friendly
#262 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
86/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
javascript, node.js
Domain
testing

Research direction

Start with tests/validate-metrics.test.mjs and the runScriptWithFixtures helper in tests/helpers.mjs, then inspect the corresponding optional-section branches in scripts/validate-metrics.mjs. Add fixture-driven cases for the fourteen listed outcomes, asserting each exit status and structured validator error, and run node --test --experimental-test-coverage to verify the suite passes.

Written by the indexing model from the issue text.

Description

quality testing

Finding

scripts/validate-metrics.mjs reports 100% lines but 30.00% branches under node --test --experimental-test-coverage. Reading tests/validate-metrics.test.mjs against the script confirms that its optional-section validation paths are never exercised. Fourteen distinct validation outcomes had no test:

  • metric with an absent id
  • metric with a null value
  • empty metrics array
  • omitted entry missing id
  • lifecycle card with a non-finite value
  • lifecycle omission missing reason
  • fully-populated valid lifecycle section
  • time series missing sourceUrl
  • series point with a non-finite value
  • fully-populated valid series section
  • breakdown whose values is not an array
  • breakdown missing its label
  • non-finite breakdown value
  • fully-populated valid breakdown section

These are the branches that only run when a metrics file carries the optional referenceArchitectureLifecycle, series, or breakdown sections. data/metrics.json is a generated artifact, so a partially-written or malformed regeneration is exactly the case these guards exist for — and exactly the case nothing currently tests.

Coverage evidence

  • Unit: node --test --experimental-test-coverage (node v26.8.2), run locally at 00b44df (current main). 55 tests pass. scripts/validate-metrics.mjs: 100.00% lines, 30.00% branches, 100% funcs.
  • End-to-end: the repository has no end-to-end or browser suite — no playwright/cypress/e2e dependency or config exists and no workflow runs one — so no end-to-end coverage can be claimed either way for this module. Tracked separately, not asserted here.
  • Measurement limitation: the two sources cannot be combined. The reported branch percentage will not move when these tests are added, because tests/helpers.mjs runScriptWithFixtures executes a temp-dir copy of the script in a subprocess, which node --test instrumentation does not attribute back to the repo path. Evidence of added coverage is therefore behavioral: each test asserts the specific exit status and the specific validator error message for its branch.

Per the coverage priority rules this is priority 2 / medium at most, and it is scoped to unit coverage only.

Recommendation

Add fixture-driven tests for each of the fourteen outcomes above, asserting exit status and the matching structured error text rather than a coverage number.

Priority

  • Impact: medium — malformed generated metrics can bypass the validator's optional-section guards untested
  • Effort: low — test-only, uses the existing runScriptWithFixtures helper

Filed by quality agent (hold-gated mode)

— hive: agent=quality backend=copilot model=claude-opus-5

Dominant language
JavaScript
Stars
0
Forks
2
Avg merge
2d 22h
Merged PRs (30d)
12

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cncf/endusers

All issues in cncf/endusers

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.