inconsistent coverage reports after migrating repos to new GitHub and codecov orgs

Aperta
#1,980 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
12/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
github, github-actions
Ambito
ci-cd, testing

Direzione di ricerca

The report is about the hosted Codecov service's per-organization behaviour, so the repo's own code is only the upload side: start with the global YAML in the issue body and the codecov/project and PR-comment outputs it is supposed to produce. The linked codecov/worker writers.py lines show the comment conditional the reporter suspects. Done would mean a maintainer confirming the cause, since the fix probably lives in the Codecov service rather than in this action.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

Describe the bug

After transferring public repositories from LizardByte to other GitHub organizations, Codecov behaves differently despite the same global YAML configuration.

All three destination organizations are using the Free plan.

Organization Repository codecov/project PR report
Themerr Themerr Missing Short, patch-only report
KokoStreamTV Koko Missing Short, patch-only report
SunshineStream Sunshine Present Full coverage comparison

One potentially relevant difference: SunshineStream has existed for a long time but was unused until these migrations. Themerr and KokoStreamTV are newly created organizations.

To Reproduce

  1. Transfer public repositories from LizardByte to the destination organizations.

  2. Configure each destination Codecov organization with the same global YAML:

    ---
    codecov:
      max_report_age: off
    
    coverage:
      status:
        project:
          default:
            target: auto
            threshold: 10%
    
    comment:
      layout: "header, reach, diff, flags, files, footer"
      behavior: default
      require_changes: false
    
    ignore:
      - "tests"
      - "third-party"
    
  3. Run GitHub Actions to upload coverage for the default branch and a pull request.

  4. Compare the resulting checks and comments:

The commit-level resolved YAML includes the configured project status and comment layout in all three repositories. No repository Codecov YAML files override the global configuration. Base and head coverage reports are available.

Expected behavior

Each public repository should receive:

  • A codecov/project check using the configured target and threshold.
  • A full PR report containing the project coverage comparison and configured layout.

Regression

The affected repositories received project coverage checks and full reports under LizardByte. The discrepancy appeared after transferring them to the newly created organizations.

Sunshine continues working after transfer to the older SunshineStream organization.

Screenshots

The linked GitHub comments show the affected and working reports.

Product Area

Coverage: GitHub status checks and pull request comments.

Versions

  • Git Host: GitHub.com
  • CI/CD: GitHub Actions
  • Codecov: Hosted service
  • Plan: Free in all three destination organizations

Commit and CI link

Affected example: Themerr PR #724

Additional context

I also tried explicitly adding this under comment:

hide_project_coverage: false

The resolved YAML confirms that this setting was applied.

On Themerr, it removed the “All tests successful. No failed tests found.” message without restoring project coverage details or the missing project check. On Sunshine, it produced no visible change; the full report already worked.

The disappearing test-success message appears consistent with this conditional in Codecov's published comment renderer:
https://github.com/codecov/worker/blob/main/services/notification/notifiers/mixins/message/writers.py#L53-L61

However, all three organizations show Free, so I am not assuming this indicates a Team subscription.

Could different defaults or internal reporting state for older versus newly created organizations explain this behavior? Is there anything that needs to be reset or corrected after the repository transfers?

Potentially related: https://github.com/codecov/codecov-action/issues/1979

That issue reports successful coverage uploads and an available codecov/patch result, while codecov/project remains running.

My situation differs: codecov/project is absent entirely, and the PR comment also omits the project coverage comparison. I’m unsure whether these share an underlying cause, but wanted to mention the similar recent report.

Lingua principale
Python
Stelle
1.7k
Fork
260
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di codecov/codecov-action

Tutte le issue di codecov/codecov-action

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.