`comment.require_changes: coverage_drop` ignores the project status threshold

Open Beginner friendly
#999 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
62/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, yaml
Domain
backend, tooling

Research direction

Start with HasEnoughRequiredChanges._check_coverage_drop in apps/worker/services/notification/notifiers/comment/conditions.py, around lines 147-159, where the threshold is read from coverage.status.project. The fix needs to look inside the named status entries such as default, not the map itself. Run the unit tests for the comment conditions, and check that a drop smaller than the configured threshold no longer posts the comment.

Written by the indexing model from the issue text.

Description

Describe the bug

The pull request comment docs say that coverage_drop "respects project status threshold, if configured (i.e. drop needs to be bigger than the configured threshold for the comment to be posted)". It does not: with a valid threshold configured, a comment is posted on any drop of more than 0.01%.

HasEnoughRequiredChanges._check_coverage_drop (conditions.py#L147-L159) reads the threshold one level too high:

project_status_config = read_yaml_field(
    comparison.comparison.current_yaml, ("coverage", "status", "project"), {}
)
threshold = 0
if isinstance(project_status_config, dict):
    threshold = Decimal(project_status_config.get("threshold", 0))

coverage.status.project is a map of named statuses (user_schema.py#L405-L418), so threshold lives under a status name such as default. project_status_config.get("threshold") therefore always returns the default 0. The shape the unit tests use, {"coverage": {"status": {"project": {"threshold": 10}}}}, is rejected by the validator: Error at ['coverage', 'status', 'project', 'threshold']: must be of ['dict', 'boolean'] type.

To Reproduce

  1. Use this codecov.yml (the validator accepts it and parses the condition to [6]):

    coverage:
      status:
        project:
          default:
            threshold: 0.5%
    comment:
      require_changes: "coverage_drop OR uncovered_patch"
    
  2. On a pull request with no existing Codecov comment and reports for both base and head, upload coverage that lowers project coverage by more than 0.01 but less than 0.5 percentage points, with no uncovered changed lines.

  3. Codecov posts the comment. In umputun/remark42#2390, coverage went from 78.58% to 78.56% (-0.03%), codecov/project passed, codecov/patch reported "Coverage not affected", and the comment was posted.

Expected behavior

No initial comment, because the project coverage drop is below the configured 0.5 percentage-point threshold and no changed lines are uncovered.

Dominant language
No language data
Stars
52
Forks
25
PR merge metrics
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from codecov/feedback

All issues in codecov/feedback

Similar issues

More Backend & API Design issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.