scancode-config.yml in an input archive is detected differently by scan_codebase and scan_single_package

Abierto
  • ¿Algo no está bien?
#2,290 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
docker, python
Área
backend

Línea de trabajo

Start by reading the config discovery logic in scanpipe/models.py and the archive extraction paths in scanpipe/pipes/scancode.py and scanpipe/pipelines/scan_single_package.py. Then review the one-level search test in scanpipe/tests/test_models.py and compare behavior for flat and single-folder archives in both pipelines. Done means the supported layout behaves consistently and any agreed warning appears when a config is found but ignored; the issue leaves those decisions open.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug

Describe the bug
A scancode-config.yml placed inside an input archive is used or ignored depending on the pipeline and on the archive layout. When it is ignored, there is no warning.

System configuration

  • Version: v38.2.0 (main at 237ea33c)
  • Docker: yes, dev stack (docker-compose.yml + docker-compose.dev.yml)
  • OS: macOS (Apple Silicon)
  • Inputs: flat.zip and wrapped.zip (attached)
  • Pipelines: scan_codebase and scan_single_package

To Reproduce
Two archives with the same content, different layout:

flat.zip:     package.json, scancode-config.yml
wrapped.zip:  my-app/package.json, my-app/scancode-config.yml

scancode-config.yml:

ignored_dependency_scopes:
  - package_type: npm
    scope: devDependencies

Run each pipeline on each archive (zips copied to /tmp in the web container):

for pipeline in scan_codebase scan_single_package; do
  for zip in flat wrapped; do
    docker compose exec -T web ./manage.py create-project "test-$pipeline-$zip" \
      --input-file /tmp/$zip.zip --pipeline $pipeline --execute
  done
done

docker compose exec -T web ./manage.py shell -c "
from scanpipe.models import Project
for p in Project.objects.filter(name__startswith='test-').order_by('name'):
    print(p.name, '->', p.get_env('ignored_dependency_scopes'))
"

Output:

test-scan_codebase-flat          -> [{'package_type': 'npm', 'scope': 'devDependencies'}]
test-scan_codebase-wrapped       -> None
test-scan_single_package-flat    -> None
test-scan_single_package-wrapped -> [{'package_type': 'npm', 'scope': 'devDependencies'}]
flat.zip wrapped.zip
scan_codebase loaded ignored
scan_single_package ignored loaded

Cause
The config is only searched in immediate codebase/ subdirectories:
https://github.com/aboutcode-org/scancode.io/blob/237ea33c9ce119e0756fd12d93a0216c47880f43/scanpipe/models.py#L894-L899

But the two pipelines extract archives to different places:

The one-level search looks intentional (#1236 and this test):
https://github.com/aboutcode-org/scancode.io/blob/237ea33c9ce119e0756fd12d93a0216c47880f43/scanpipe/tests/test_models.py#L746-L753

Most real source archives (GitHub release tarballs, sdists, npm tarballs) have a single top folder, so with scan_codebase their config is silently ignored.

Expected behavior
The same archive should behave the same way in both pipelines. Questions:

  1. Which layout should be supported: top level, single top folder, or both?
  2. Should a warning be logged when a scancode-config.yml exists in the codebase but is not used?

Related
#2215 may be a real case of this (config inside a source archive). Note: in my test jest is still listed as a dependency even when the config is loaded; that is the separate package_type issue fixed in #2218.

I'm happy to work on a fix once the expected behavior is agreed.

AI disclosure: I used Claude Code to help explore the codebase and investigate. I ran the reproduction above myself and checked the results.

Lenguaje dominante
Python
Estrellas
225
Forks
216
Merge medio
1 d 59 min
PR fusionados (30 d)
13

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de aboutcode-org/scancode.io

Todos los issues de aboutcode-org/scancode.io

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.