scancode-config.yml in an input archive is detected differently by scan_codebase and scan_single_package

オープン
  • 何か問題がありますか？
#2,290 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
48/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
docker, python
領域
backend

調査の方向性

Start by reading the config discovery logic in scanpipe/models.py and the archive extraction paths in scanpipe/pipes/scancode.py and scanpipe/pipelines/scan_single_package.py. Then review the one-level search test in scanpipe/tests/test_models.py and compare behavior for flat and single-folder archives in both pipelines. Done means the supported layout behaves consistently and any agreed warning appears when a config is found but ignored; the issue leaves those decisions open.

索引モデルが issue の本文から書いたものです。

説明

bug

Describe the bug
A scancode-config.yml placed inside an input archive is used or ignored depending on the pipeline and on the archive layout. When it is ignored, there is no warning.

System configuration

  • Version: v38.2.0 (main at 237ea33c)
  • Docker: yes, dev stack (docker-compose.yml + docker-compose.dev.yml)
  • OS: macOS (Apple Silicon)
  • Inputs: flat.zip and wrapped.zip (attached)
  • Pipelines: scan_codebase and scan_single_package

To Reproduce
Two archives with the same content, different layout:

flat.zip:     package.json, scancode-config.yml
wrapped.zip:  my-app/package.json, my-app/scancode-config.yml

scancode-config.yml:

ignored_dependency_scopes:
  - package_type: npm
    scope: devDependencies

Run each pipeline on each archive (zips copied to /tmp in the web container):

for pipeline in scan_codebase scan_single_package; do
  for zip in flat wrapped; do
    docker compose exec -T web ./manage.py create-project "test-$pipeline-$zip" \
      --input-file /tmp/$zip.zip --pipeline $pipeline --execute
  done
done

docker compose exec -T web ./manage.py shell -c "
from scanpipe.models import Project
for p in Project.objects.filter(name__startswith='test-').order_by('name'):
    print(p.name, '->', p.get_env('ignored_dependency_scopes'))
"

Output:

test-scan_codebase-flat          -> [{'package_type': 'npm', 'scope': 'devDependencies'}]
test-scan_codebase-wrapped       -> None
test-scan_single_package-flat    -> None
test-scan_single_package-wrapped -> [{'package_type': 'npm', 'scope': 'devDependencies'}]
flat.zip wrapped.zip
scan_codebase loaded ignored
scan_single_package ignored loaded

Cause
The config is only searched in immediate codebase/ subdirectories:
https://github.com/aboutcode-org/scancode.io/blob/237ea33c9ce119e0756fd12d93a0216c47880f43/scanpipe/models.py#L894-L899

But the two pipelines extract archives to different places:

The one-level search looks intentional (#1236 and this test):
https://github.com/aboutcode-org/scancode.io/blob/237ea33c9ce119e0756fd12d93a0216c47880f43/scanpipe/tests/test_models.py#L746-L753

Most real source archives (GitHub release tarballs, sdists, npm tarballs) have a single top folder, so with scan_codebase their config is silently ignored.

Expected behavior
The same archive should behave the same way in both pipelines. Questions:

  1. Which layout should be supported: top level, single top folder, or both?
  2. Should a warning be logged when a scancode-config.yml exists in the codebase but is not used?

Related
#2215 may be a real case of this (config inside a source archive). Note: in my test jest is still listed as a dependency even when the config is loaded; that is the separate package_type issue fixed in #2218.

I'm happy to work on a fix once the expected behavior is agreed.

AI disclosure: I used Claude Code to help explore the codebase and investigate. I ran the reproduction above myself and checked the results.

主要言語
Python
スター
225
フォーク
216
平均マージ
1日 59分
マージ済み PR（30日）
13

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

aboutcode-org/scancode.io のほかの issue

aboutcode-org/scancode.io の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。