Coverage is measured differently locally and on CI, hiding 68 untested app files
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 35/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
- 领域
- backend, testing-qa
调研方向
从 config/environments/test.rb 和 spec/spec_helper.rb 开始，然后运行 issue 中描述的本地和 CI 风格的 bundle exec rspec 命令。在决定如何补充测试之前，检查纳入范围的 app 文件及其覆盖率。完成的标准是本地覆盖率和 CI 覆盖率衡量相同的应用范围，并且新纳入范围但尚未测试的代码已得到处理。
由索引模型根据 Issue 内容生成。
描述
The backend coverage figure means different things locally and on CI, and the local one is the misleading one. Reproduced exactly, not inferred.
$ bundle exec rspec 3772 / 3948 LOC (95.54%)
$ CI=true bundle exec rspec 4193 / 4856 LOC (86.35%) # identical to CI's report
Cause
Two settings that are each fine alone:
config/environments/test.rb—config.eager_load = ENV["CI"].present?. This is the stock Rails default, verbatim from therailtiesapp template; nobody misconfigured it.spec/spec_helper.rb— a bareSimpleCov.start, with no profile and therefore notrack_files.
Without track_files, SimpleCov measures only files that were actually loaded. GitHub Actions always sets CI=true, so Rails eager-loads all of app/; locally it does not, so only the files the specs happen to touch are loaded, and only those are measured.
What that hides
The delta accounts for itself exactly — 4856 − 3948 and 4193 − 3772:
70 files, 908 relevant lines, 421 covered. 68 of the 70 are app/ files, so 38% of the application is never loaded by the suite at all.
Those files are not reported as 0% locally. They are absent from the report. The local denominator is defined by what the tests already reach, which makes 95.54% close to circular.
The CI number is not trustworthy either, for a different reason: spec/ files are 1952 of its 4856 lines — 40% — at 98.51% covered, because the test_frameworks filter only ships with SimpleCov's rails profile, which is not loaded. The tests are grading themselves.
Honest figure for application code: 77.09%.
The 46% those 70 files appear to have on CI is pure load-time execution. Line by line on app/jobs/same_trackables_job.rb (18.2% on CI, invisible locally), the only covered lines are class, include Sidekiq::Worker, and two defs. Every method body is 0. The file is completely untested.
The obvious fix does not work
Adding track_files "{app,lib}/**/*.rb" makes it worse — the two environments then disagree on the denominator as well, because SimpleCov's static LinesClassifier counts relevant lines differently than runtime Coverage:
track_files + CI=true -> 4193 / 4856 (86.35%) unchanged
track_files + local -> 3772 / 5367 (70.28%)
I measured the filter options too. Every one leaves local ≠ CI, and filtering spec/ widens the gap (9.2pp → 15.2pp), since spec/ is the one bucket both environments agree on:
| Config | eager (CI) | lazy (local) |
|---|---|---|
| current | 86.35% | 95.54% |
filter spec/ |
78.17% | 92.64% |
rails profile equivalent |
77.09% | 92.31% |
eager_load is the whole story. No SimpleCov setting reconciles it.
The part that actually bites
Eager loading is in CI specifically to catch autoload and NameError breakage. Gating it on CI means that entire class of bug cannot fail locally — only on CI, after a push.
Proposed fix
config.eager_load = true in test, unconditionally. Measured cost: ~0.5s on boot, 2.7s → 3.2s.
Then backfill the coverage it exposes, which is the larger half of the work.
Separately worth deciding, but deliberately not bundled in: SimpleCov.start "rails" would stop the tests grading themselves and report ~77%. That is a policy change that visibly drops the headline number and should be a conscious call, not smuggled in behind a defect fix.
🤖 Generated with Claude Code
- 主要语言
- Ruby
- 星标
- 50
- 派生
- 21
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
rubyforgood/Flaredown 的其他 Issue
-
Four API faults found while backfilling coverage, documented but unfixed可能已有人在做 @suttondemlong 于 47 天前认领。 未关闭language:ruby type:bug
难度 4/5 3-5 天 新手友好度 48/100
rubyforgood/Flaredown#913 · 1 条评论 ·
-
type:infrastructure
难度 3/5 1-2 天 新手友好度 76/100
rubyforgood/Flaredown#904 ·
-
type:infrastructure
难度 4/5 3-5 天 新手友好度 45/100
rubyforgood/Flaredown#888 ·
-
type:infrastructure
难度 4/5 3-5 天 新手友好度 38/100
rubyforgood/Flaredown#887 ·
-
type:infrastructure
难度 4/5 3-5 天 新手友好度 48/100
rubyforgood/Flaredown#886 ·
查看 rubyforgood/Flaredown 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 88/100
TheOdinProject/curriculum#31493 ·
维护者通常 1 天内回复
-
enhancement
难度 1/5 1 小时以内 新手友好度 62/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 76/100
rmosolgo/graphql-ruby#5843 ·
维护者通常 1 天内回复
-
难度 1/5 1-3 小时 新手友好度 66/100
betagouv/conseillers-entreprises#4770 ·
维护者通常 1 天内回复