scancode-config.yml in an input archive is detected differently by scan_codebase and scan_single_package

Aperta
  • Qualcosa non va?
#2,290 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
docker, python
Ambito
backend

Direzione di ricerca

Start by reading the config discovery logic in scanpipe/models.py and the archive extraction paths in scanpipe/pipes/scancode.py and scanpipe/pipelines/scan_single_package.py. Then review the one-level search test in scanpipe/tests/test_models.py and compare behavior for flat and single-folder archives in both pipelines. Done means the supported layout behaves consistently and any agreed warning appears when a config is found but ignored; the issue leaves those decisions open.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

Describe the bug
A scancode-config.yml placed inside an input archive is used or ignored depending on the pipeline and on the archive layout. When it is ignored, there is no warning.

System configuration

  • Version: v38.2.0 (main at 237ea33c)
  • Docker: yes, dev stack (docker-compose.yml + docker-compose.dev.yml)
  • OS: macOS (Apple Silicon)
  • Inputs: flat.zip and wrapped.zip (attached)
  • Pipelines: scan_codebase and scan_single_package

To Reproduce
Two archives with the same content, different layout:

flat.zip:     package.json, scancode-config.yml
wrapped.zip:  my-app/package.json, my-app/scancode-config.yml

scancode-config.yml:

ignored_dependency_scopes:
  - package_type: npm
    scope: devDependencies

Run each pipeline on each archive (zips copied to /tmp in the web container):

for pipeline in scan_codebase scan_single_package; do
  for zip in flat wrapped; do
    docker compose exec -T web ./manage.py create-project "test-$pipeline-$zip" \
      --input-file /tmp/$zip.zip --pipeline $pipeline --execute
  done
done

docker compose exec -T web ./manage.py shell -c "
from scanpipe.models import Project
for p in Project.objects.filter(name__startswith='test-').order_by('name'):
    print(p.name, '->', p.get_env('ignored_dependency_scopes'))
"

Output:

test-scan_codebase-flat          -> [{'package_type': 'npm', 'scope': 'devDependencies'}]
test-scan_codebase-wrapped       -> None
test-scan_single_package-flat    -> None
test-scan_single_package-wrapped -> [{'package_type': 'npm', 'scope': 'devDependencies'}]
flat.zip wrapped.zip
scan_codebase loaded ignored
scan_single_package ignored loaded

Cause
The config is only searched in immediate codebase/ subdirectories:
https://github.com/aboutcode-org/scancode.io/blob/237ea33c9ce119e0756fd12d93a0216c47880f43/scanpipe/models.py#L894-L899

But the two pipelines extract archives to different places:

The one-level search looks intentional (#1236 and this test):
https://github.com/aboutcode-org/scancode.io/blob/237ea33c9ce119e0756fd12d93a0216c47880f43/scanpipe/tests/test_models.py#L746-L753

Most real source archives (GitHub release tarballs, sdists, npm tarballs) have a single top folder, so with scan_codebase their config is silently ignored.

Expected behavior
The same archive should behave the same way in both pipelines. Questions:

  1. Which layout should be supported: top level, single top folder, or both?
  2. Should a warning be logged when a scancode-config.yml exists in the codebase but is not used?

Related
#2215 may be a real case of this (config inside a source archive). Note: in my test jest is still listed as a dependency even when the config is loaded; that is the separate package_type issue fixed in #2218.

I'm happy to work on a fix once the expected behavior is agreed.

AI disclosure: I used Claude Code to help explore the codebase and investigate. I ran the reproduction above myself and checked the results.

Lingua principale
Python
Stelle
225
Fork
216
Merge medio
1g 59m
PR unite (30g)
13

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di aboutcode-org/scancode.io

Tutte le issue di aboutcode-org/scancode.io

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.