Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Automatically ratchet up coverage thresholds instead of hand-maintaining them

Đang mở
  • Có gì không ổn?
#1,453 0 bình luận 0 reaction 1 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@YuryShkoda đang làm issue này rồi.

Từ ngày 3/7/2026.

Đánh giá

Issue này chưa được đánh giá.

Mô tả

refactoring

Problem

jest.config.js hardcodes coverageThreshold.global as fixed numbers:

coverageThreshold: {
  global: {
    statements: 73,
    branches: 60.55,
    functions: 73.54,
    lines: 74
  }
}

These only get updated when someone notices npm run test:mocked failing (threshold not met) and manually edits the numbers — usually while already deep in an unrelated PR. This has two failure modes:

  • Silent stagnation: when coverage improves, nothing bumps the threshold up, so the safety net stays looser than it could be indefinitely.
  • Noisy, unrelated diffs: when coverage regresses, whoever's PR happens to trip the threshold has to manually figure out the right new number and edit CI-adjacent config as a side quest to their actual change.

We want this to self-maintain, the same way semantic-release (already used in this repo's npmpublish.yml) self-maintains the package version — no human edits jest.config.js by hand for this again.

Goal

Coverage thresholds should only ever move in the safe direction (up when real coverage improves) automatically, and a genuine regression should still fail CI clearly - never be silently masked.

Two viable approaches — needs a decision before implementation

Option A: custom script + CI auto-commit (self-hosted, more code to own)
  • A script (e.g. scripts/updateCoverageThresholds.js) runs after npm run test:mocked and reads coverage/coverage-summary.json.
  • For each of the four metrics, if actual coverage > current jest.config.js threshold, rewrite jest.config.js with the new value rounded down slightly (e.g. -0.5pt buffer) rather than an exact match, so a trivial/flaky variance between runs doesn't immediately fail the next build.
  • Never auto-lowers a threshold — only ratchets up. A real regression still fails the existing test:mocked step exactly as it does today, forcing a human decision (fix the coverage, or consciously accept + explain the drop in the PR).
  • Runs as a step in a workflow triggered on push to main (mirroring npmpublish.yml's trigger) — after a PR has already merged, not as part of PR checks — and pushes a bot commit like chore: bump coverage thresholds directly to main, the same way semantic-release pushes its version-bump commit.
  • Needs: write access from the Action to main (existing GITHUB_TOKEN with contents:write, or a PAT if branch protection requires it), and a guard so the bot's own commit doesn't re-trigger itself in a loop.
Option B: external coverage service (Codecov / Coveralls) - likely less code to build and maintain
  • These already solve exactly this class of problem: instead of a static number in the repo, they compare each PR's coverage against the base branch and post a status check like "coverage decreased by X% - failing" or "coverage increased - no threshold to maintain at all."
  • No jest.config.js numbers to bump, no bot-commit workflow to build/maintain, PR-level annotations showing exactly which lines lost coverage, and a badge for the README as a bonus.
  • Trade-off: adds a third-party dependency (an account + a small config file, e.g. codecov.yml), and CI needs to upload the coverage report (coverage/lcov.info) to the service each run.
  • This repo already uses artiomtr/jest-coverage-report-action in run-tests.yml for PR coverage comments - worth checking whether it (or a config tweak to it) already covers this need before adding another service.

Suggested next step

  • Decide between Option A and Option B (see trade-offs above) - or confirm neither and reconsider artiomtr/jest-coverage-report-action's existing capabilities first, since it's already wired into run-tests.yml.
  • If Option A: write scripts/updateCoverageThresholds.js, wire it into a new (or existing, post-merge) workflow step, decide the exact buffer value, and add the loop-prevention guard for the bot's own commits.
  • If Option B: pick a service, add the upload step to run-tests.yml, configure its ratcheting/regression rules, and remove the now-redundant coverageThreshold block from jest.config.js.
  • Either way: document the chosen mechanism (e.g. in CONTRIBUTING.md or a code comment near coverageThreshold) so contributors know they should never hand-edit the threshold numbers going forward.

Non-negotiable

Whichever option is picked, a genuine coverage regression must still fail CI loudly - this issue is about removing manual upkeep of the good direction (raising the bar), not about loosening enforcement.

Ngôn ngữ chính
TypeScript
Star
41
Fork
6
Merge trung bình
7 giờ 12 phút
Pull request đã merge (30 ngày)
26

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của sasjs/cli

Tất cả issue của sasjs/cli

Issue tương tự

Thêm issue về TypeScript

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.