Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Coverage is measured differently locally and on CI, hiding 68 untested app files

Ouverte
  • Quelque chose ne va pas ?
#911 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

@suttondemlong y travaille déjà.

Depuis le 9/9/2026.

  • #912 par @suttondemlong — ouverte

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
35/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
rails, ruby
Domaine
backend, testing-qa

Piste de recherche

Commencez par config/environments/test.rb et spec/spec_helper.rb, puis exécutez les commandes bundle exec rspec locales et de type CI décrites dans l’issue. Examinez les fichiers de l’application exposés et leur couverture avant de décider comment compléter les tests. C’est terminé lorsque la couverture locale et celle de CI mesurent le même périmètre de l’application et que le code nouvellement exposé qui n’est pas testé est pris en charge.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

language:ruby type:bug

The backend coverage figure means different things locally and on CI, and the local one is the misleading one. Reproduced exactly, not inferred.

$ bundle exec rspec              3772 / 3948 LOC (95.54%)
$ CI=true bundle exec rspec      4193 / 4856 LOC (86.35%)   # identical to CI's report

Cause

Two settings that are each fine alone:

  • config/environments/test.rb — config.eager_load = ENV["CI"].present?. This is the stock Rails default, verbatim from the railties app template; nobody misconfigured it.
  • spec/spec_helper.rb — a bare SimpleCov.start, with no profile and therefore no track_files.

Without track_files, SimpleCov measures only files that were actually loaded. GitHub Actions always sets CI=true, so Rails eager-loads all of app/; locally it does not, so only the files the specs happen to touch are loaded, and only those are measured.

What that hides

The delta accounts for itself exactly — 4856 − 3948 and 4193 − 3772:

70 files, 908 relevant lines, 421 covered. 68 of the 70 are app/ files, so 38% of the application is never loaded by the suite at all.

Those files are not reported as 0% locally. They are absent from the report. The local denominator is defined by what the tests already reach, which makes 95.54% close to circular.

The CI number is not trustworthy either, for a different reason: spec/ files are 1952 of its 4856 lines — 40% — at 98.51% covered, because the test_frameworks filter only ships with SimpleCov's rails profile, which is not loaded. The tests are grading themselves.

Honest figure for application code: 77.09%.

The 46% those 70 files appear to have on CI is pure load-time execution. Line by line on app/jobs/same_trackables_job.rb (18.2% on CI, invisible locally), the only covered lines are class, include Sidekiq::Worker, and two defs. Every method body is 0. The file is completely untested.

The obvious fix does not work

Adding track_files "{app,lib}/**/*.rb" makes it worse — the two environments then disagree on the denominator as well, because SimpleCov's static LinesClassifier counts relevant lines differently than runtime Coverage:

track_files + CI=true   ->  4193 / 4856  (86.35%)   unchanged
track_files + local     ->  3772 / 5367  (70.28%)

I measured the filter options too. Every one leaves local ≠ CI, and filtering spec/ widens the gap (9.2pp → 15.2pp), since spec/ is the one bucket both environments agree on:

Config eager (CI) lazy (local)
current 86.35% 95.54%
filter spec/ 78.17% 92.64%
rails profile equivalent 77.09% 92.31%

eager_load is the whole story. No SimpleCov setting reconciles it.

The part that actually bites

Eager loading is in CI specifically to catch autoload and NameError breakage. Gating it on CI means that entire class of bug cannot fail locally — only on CI, after a push.

Proposed fix

config.eager_load = true in test, unconditionally. Measured cost: ~0.5s on boot, 2.7s → 3.2s.

Then backfill the coverage it exposes, which is the larger half of the work.

Separately worth deciding, but deliberately not bundled in: SimpleCov.start "rails" would stop the tests grading themselves and report ~77%. That is a policy change that visibly drops the headline number and should be a conscious call, not smuggled in behind a defect fix.

🤖 Generated with Claude Code

Langage dominant
Ruby
Étoiles
50
Forks
21
Aucune PR mergée en 30 j

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de rubyforgood/Flaredown

Toutes les issues de rubyforgood/Flaredown

Issues similaires

Plus d'issues Ruby

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.