match_unmapped_resources ignores matched_extensions and re-sends those files to PurlDB

Open
#2,338 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@AradhyaTiwari10 is already working on this.

Since Nov 16, 2026.

  • #2339 by @AradhyaTiwari10 — open

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
25/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python
Domain
backend, databases

Research direction

Read match_unmapped_resources() in scanpipe/pipes/d2d.py and the corresponding tests in test_d2d.py. The issue reports that the result of .exclude() is discarded; confirm which resources reach the mocked PurlDB matcher when matched_extensions=[".java"]. Done means only the unmatched extension reaches the matcher and the test passes; a linked pull request is already open.

Written by the indexing model from the issue text.

Description

Describe the bug

match_unmapped_resources() in scanpipe/pipes/d2d.py is supposed to skip the extensions that were already looked up in PurlDB by the match_resources_to_purldb step (matched_extensions=self.ecosystem_config.matchable_resource_extensions), but the filter is never applied:

if matched_extensions:
    to_unmapped.exclude(extension__in=matched_extensions)

The result of .exclude() is discarded, so to_unmapped still contains every file with a matched extension (for example all the .class files of a Java project) and they are sent to PurlDB a second time.

System configuration

  • ScanCode.io main (39acd55)
  • Local install, macOS
  • Pipeline: map_deploy_to_develop

To Reproduce

  1. Create a project with two to/ files that have no status, for example Baz.java and expected.json.
  2. Call d2d.match_unmapped_resources(project, matched_extensions=[".java"], logger=print) with _match_purldb_resources mocked.
  3. The log says Mapping 2 to/ resources with empty status in PurlDB using SHA1 and Baz.java is passed to the PurlDB matcher.

Expected behavior

Only expected.json is sent to PurlDB (Mapping 1 to/ resources ...). Files with a matched extension still get the requires-review status at the end of the step, as they do today.

Proposed fix: assign the result, to_unmapped = to_unmapped.exclude(extension__in=matched_extensions), plus a unit test in test_d2d.py that calls match_unmapped_resources with matched_extensions and checks which resources reach the matcher. I have the change and the test ready locally and can open a PR.

Dominant language
Python
Stars
226
Forks
222
Avg merge
1d 5h
Merged PRs (30d)
32

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from aboutcode-org/scancode.io

All issues in aboutcode-org/scancode.io

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.