Recognize native build systems (Kbuild/Kconfig, CMake, Meson, Bazel/Buck, autotools, Make) as static component evidence
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 25/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- cmake, go
- Ambito
- build-system
Direzione di ricerca
Start with the proposed new pkg/buildsys/ package and read the existing manifest-recognition implementation referenced as #5 to understand how adapters represent components and evidence. The first listed task is the Kbuild/Kconfig adapter; its acceptance cases are outlined in pkg/buildsys/kbuild_test.go, including object types, conditional symbols, descent, and comment handling. Done means the adapter and its tests meet those cases; the issue also describes a much broader set of adapters, fixtures, and integration work.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
User outcome
- The map of a C, C++ or systems codebase shows its real structure: build targets, libraries, executables, kernel subsystems and modules, and configuration options, with evidence.
- Today such a map shows nothing, because those projects have no package-manager manifest.
Why
- The kernel. Measured on the #94 candidate: the Linux kernel map had one component (
tools/net/ynl, a small Python tool) and zero deployables. The kernel is the first repository many skeptical readers will try, and "one Python component" reads as broken. - The gap is structural. Component recognition today covers npm, Go, Python/uv, Cargo, Maven, Gradle and .NET. Those are ecosystems with declarative manifests. Most systems code declares its structure in build files instead: Kbuild/Kconfig, CMake, Meson, Bazel/Buck, autotools, plain Makefiles.
- Static parsing is enough. These can be read as data for a large useful subset without evaluating arbitrary build logic, just as #5 does for manifests. For Bazel/Buck (Starlark) there's a mature parse-only Go library: bazelbuild/buildtools
buildpackage. That leverages existing work rather than re-implementing it.
Proposal
Static, bounded adapters that produce components, membership and local-dependency edges, each with declared_config evidence and honest partial coverage wherever evaluation would be needed:
- Kbuild/Kconfig.
obj-y/obj-m/obj-$(CONFIG_X)lines produce built-in vs module objects per directory, and subdirectory descent (obj-y += dir/) produces containment.config/menuconfigsymbols are recorded with their defining file.- Conditional objects keep their controlling
CONFIG_*symbol as a condition. Nothing is evaluated. - Subsystem components are derived from the directory structure as declared by Kbuild descent, not from guesswork.
- CMake.
project(),add_executable,add_library(with STATIC/SHARED/INTERFACE),add_subdirectoryand literaltarget_link_librariesproduce targets and edges.- Variables, generator expressions,
include()of computed paths and functions are recorded as unresolved references.
- Meson.
project,executable,library/shared_library/static_library,subdiranddependency()names. - Bazel/Buck(2). Parse BUILD and BUCK files with buildtools. Rule calls with literal
name,srcsanddepsproduce targets and local edges; macros and load()ed rules are recorded as unresolved. - Autotools and Make.
configure.acAC_INITproduces project identity;Makefile.ambin_PROGRAMS/lib_LTLIBRARIESproduce targets.- Plain Makefiles contribute only explicit, literal target names from rules. Everything else is
unknown, deliberately conservative.
- Cross-checks (see the #94 finding on overclaimed
complete): a directory with C/C++ sources and build files, but no recognized target, degrades thecomponentsquestion withunrecognized_build_declarations.
Tasks
- Kbuild/Kconfig adapter first (the highest-visibility demo), including a scalable directory-descent graph for about 3,000 Makefiles.
- CMake adapter (a literal subset, with an unresolved-reference taxonomy).
- Bazel/Buck adapter via buildtools.
- Meson and autotools adapters.
- Plain-Make literal targets.
- Summary rendering for systems repos: top subsystems by size and number of targets, not a flat list of 3,000 directories.
Test plan
Unit tests
New package pkg/buildsys/ (new) with one _test.go file per adapter:
pkg/buildsys/kbuild_test.go(new): table-driven cases forobj-y,obj-m,obj-$(CONFIG_X)lines (conditional object records itsCONFIG_*symbol, not a resolved value); subdirectory descent viaobj-y += dir/;config/menuconfigsymbol extraction; skipping#-comment lines; a directory with only unrecognized assignment forms produces zero objects and zero subsystems; a Makefile with 3,000 entries parses in under 500 ms (time assertion).pkg/buildsys/cmake_test.go(new):project(),add_executable,add_librarywith STATIC/SHARED/INTERFACE variants,add_subdirectory,target_link_librarieswith literal names; a variable reference (${VAR}) in a target name produces an unresolved-reference record, not a target; ainclude()of a computed path is recorded as an unresolved include; generator expressions ($<TARGET_FILE:t>) are rejected as unresolved;if()/else()blocks are not evaluated — all branches contribute targets as partial candidates withconditionalflag.pkg/buildsys/bazel_test.go(new):cc_binary,cc_library,go_binary,py_libraryrule calls with literalname,srcsanddeps; aload()that introduces a macro is recorded as an unresolved rule origin; aglob()insrcsis recorded as an unresolved source set;depscontaining//externallabels produce cross-workspace edges, which are dropped with anexternal_depreason.pkg/buildsys/meson_test.go(new):project(),executable(),library(),shared_library(),static_library(),subdir(),dependency()with a literal name; adependency()with a computed name is unresolved.pkg/buildsys/autotools_test.go(new):AC_INITfromconfigure.acproduces project identity;bin_PROGRAMS,lib_LTLIBRARIESfromMakefile.amproduce targets; aMakefile.amwith onlySUBDIRSproduces subdirectory containment edges.pkg/buildsys/make_test.go(new): explicit literal target names from top-level rules; phony targets (.PHONY) are excluded from components; pattern rules (e.g.,%.o: %.c) are not reported as targets; an included Makefile at a computed path is recorded as an unresolved include.pkg/buildsys/cross_test.go(new): a C directory with.cfiles and no recognized build file degrades tounrecognized_build_declarationscoverage on thecomponentsquestion.
Hand-labeled ground truth and fixtures (tied to #75)
Ground truth is hand-labeled once and committed as JSON; any deviation blocks the PR.
Linux kernel (Kbuild/Kconfig):
- New
tests/build_adapters/labels/kernel/(new). Labels cover:- the 20 top-level subsystem directories (
arch/,drivers/,fs/,net/,mm/,kernel/,sound/,security/,crypto/,lib/,block/,ipc/,init/,firmware/,Documentation/,tools/,scripts/,include/,samples/,usr/); - a hand-labeled sample of 50 directories'
obj-yandobj-msets, drawn fromdrivers/usb/,net/ipv4/,fs/ext4/,arch/x86/kernel/,security/selinux/(representative of conditional and unconditional objects); - 10
config/menuconfigsymbol definitions fromKconfigfiles atarch/x86/Kconfig,drivers/Kconfig,net/Kconfig.
- the 20 top-level subsystem directories (
- Precision/recall gate: ≥ 95% precision and ≥ 90% recall on the labeled
obj-y/obj-msets; 100% recall on top-level subsystem containment edges. - The kernel source tree is not committed;
tests/build_adapters/fetch_repos.py(new) fetches a pinned tag (v6.10) under a.gitignored cache directory (per #88).
CMake (abseil-cpp or the CMake project itself):
- Labels in
tests/build_adapters/labels/cmake/(new): top-levelproject()identity, alladd_libraryandadd_executabletargets at depth ≤ 2, and 10target_link_librariesedges with literal deps. Pinned commit. - Fetched by
tests/build_adapters/fetch_repos.py, not committed.
Bazel (abseil-cpp with Bazel BUILD files):
- Labels in
tests/build_adapters/labels/bazel/(new):cc_librarytargets fromabsl/strings/BUILD.bazelandabsl/base/BUILD.bazel, plusdepsedges that are literal local labels. Pinned commit.
Meson (systemd or libpng):
- Labels in
tests/build_adapters/labels/meson/(new): top-levelproject(), first-levelsubdir()calls,executableandlibrarytargets with literal names. Pinned tag.
Autotools (GNU coreutils):
- Labels in
tests/build_adapters/labels/autotools/(new):AC_INITidentity,bin_PROGRAMStargets from the top-levelMakefile.am. Pinned tag.
Adversarial fixtures
Committed under tests/build_adapters/fixtures/adversarial/ (new). Each is a minimal synthetic directory tree, no binaries, generated inline as strings in a gen_adversarial.py (new) script:
- macro-heavy CMake: 50
include()calls with computed paths, no literal targets → all unresolved, no crash. - generated Makefiles: a
Makefilethatincludes a pattern-generated file → unresolved include, zero targets. - recursive Kbuild: a chain of 200
obj-y += dir/descents → bounded at the configured depth limit, partial coverage. - large Kconfig: 5,000
configsymbols in a single file → parses without OOM, counts reported accurately. - non-UTF-8 source file: a
CMakeLists.txtwith a Latin-1 encoded comment → parser skips the bad bytes, reports unresolved, no panic. - duplicate targets: two
add_library(foo ...)calls → second is flagged as a duplicate with a diagnostic, first is retained.
Differential oracles (tied to #84)
Hermetic containers (new Dockerfiles in tests/build_adapters/docker/, --network=none at runtime), run on workflow_dispatch and on PRs touching pkg/buildsys/:
- CMake file API (
dircue-cmake-oracle:cmake-3.30, new): runscmake -S . -B /tmp/build -DCMAKE_EXPORT_COMPILE_COMMANDS=ONfollowed by reading.cmake/api/v1/reply/to extract targets; dircue's literal-subset target list must be a subset of the file-API list (precision gate ≥ 95%); recall is expected to be lower (macros and generator expressions produce API targets dircue can't see, which is correct behavior). The oracle never executes user build code — it reads only the file-API JSON reply, which CMake itself writes.- Oracle images are pinned (tag
cmake-3.30); the Dockerfile is the source of truth for the version. - Labeled repo: abseil-cpp at the pinned commit.
- Oracle images are pinned (tag
- Bazel query (
dircue-bazel-oracle:bazel-7.4, new):bazel query //... --output=label(in a sandbox with--noenable_workspace); dircue's literal target list must be a subset of the query output. Same precision/recall framing.- Labeled repo: abseil-cpp at the pinned commit.
- Note: Bazel downloads dependencies during query even with
--noenable_workspace; the container image pre-fetches dependencies at build time so the runtime container has--network=none.
- Kbuild cross-check (no container needed): compare dircue's
obj-y/obj-mset for the labeled 50 directories against the hand labels; this is purely a label-match check, not a process-based oracle, since Kbuild can only be fully evaluated by the kernel build system.
Fuzz targets (tied to #87)
- New
pkg/buildsys/fuzz_test.go(new):FuzzKbuildLine(f *testing.F),FuzzCMakeLine(f *testing.F),FuzzBazelFile(f *testing.F),FuzzMesonFile(f *testing.F). Seeds: one small real file per system from the labeled repos. Properties: never panics, output is deterministic for identical input, byte count of unresolved references is non-negative. - Persisted corpora per #87; not in CI by default.
Mutation-testing focus (tied to #86)
- The Kbuild
obj-y/obj-mdiscriminator: mutation of the string comparison silently conflates built-in and module objects. Every labeled directory has at least oneobj-mentry to catch this. - The CMake conditional-flag logic: mutation that drops the
conditionalflag causes phantom unconditional targets to be reported. - The tree-entry sort order in the Bazel
depsedge list: dircue's output must be deterministic; mutation of sort direction changes the output but may not fail without a determinism assertion.
Performance and resource checks (tied to #89/#90)
- Deterministic counters (new):
buildsys_files_parsed,buildsys_targets_found,buildsys_unresolved_refsadded to the map summary for any repo with recognized build files; gated non-negative in PR CI for the labeled repos. - Kernel Kbuild parse time:
make bench-kbuild(new make target) runspkg/buildsys/BenchmarkKbuildover a 3,000-Makefile synthetic tree; assert completes in under 30 s. - On-demand benchmark
make bench-buildsys(new) runs all adapters over the labeled repos and reports targets/s and files/s.
E2E script
tests/build_adapters/e2e.py(new). For each labeled repo:- Runs
dircue mapand extracts the component list and dependency edges. - Compares against the hand labels: precision and recall per adapter.
- Where a container oracle exists, runs it and compares against oracle output.
- Logs to
build_adapters_results.jsonl:- fields:
repo,adapter,files_parsed,targets_found,unresolved_refs,label_precision,label_recall,oracle_precision,oracle_recall,wall_ms,rss_kb.
- fields:
- Prints a scoreboard table with a pass/fail per adapter.
- Exits non-zero if any adapter's label precision is below 90% or recall below 80%.
- Runs
CI placement
- Unit tests (
pkg/buildsys/...) run in the existingtestjob on every PR (fast, no network, no container). - Adversarial fixture tests run in the existing
testjob on every PR (deterministic, small). - Oracle containers (
tests/build_adapters/e2e.pywith container steps) run onworkflow_dispatchand on PRs touchingpkg/buildsys/; new jobbuildsys-oraclein.github/workflows/ci.yml(new job),--network=none, 15-minute timeout. - Kernel-scale Kbuild benchmark is
workflow_dispatch/make bench-kbuildonly (large fetch).
Regression fences
- Any PR that reduces
label_precisionbelow 90% orlabel_recallbelow 80% on a previously passing adapter is a regression blocker. - A directory with recognized C/C++ sources and build files that produces zero targets must emit the
unrecognized_build_declarationsdiagnostic; agrep-based check in thepreflightCI job (extend existing) enforces this. - No build tool is ever invoked by dircue; a
strace/dtracecheck in the oracle container (rundircueunderstrace -e execve) asserts zeroexecvecalls tocmake,make,bazel,meson, orninja.
Risks
- Overreach into evaluation. Keep a strict literal subset, and make "unresolved" first-class.
- Kernel noise. The kernel has an enormous number of conditional objects. The summary needs aggregation to stay readable.
Success criteria
- The kernel map shows its subsystems and module structure with evidence.
- Measured precision and recall on labeled systems repos are published.
- No build tool is ever executed by dircue.
Related: #5, #74, #75, #54, #23. Related idea issues: #103, #107.
Sequencing: Independent; do Kbuild first. Measured accuracy is published through #107.
This proposal came out of an idea-wizard session on 2026-09-23. It isn't a release commitment, and the milestone is deliberately unset so the owner can decide per idea. It must preserve dircue's contracts:
- deterministic, offline output, with no execution of inspected content;
- the legacy Linguist-compatible CLI and output unchanged;
unknownnever rendered as absent;- no provider findings or verdicts imported;
- CI event-driven only (no
schedule:).
- Lingua principale
- Go
- Stelle
- 0
- Fork
- 0
- Merge medio
- 5h 8m
- PR unite (30g)
- 54
Preparare l'ambiente
- Include un Dockerfile o un file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di war-and-code/dircue
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 86/100
war-and-code/dircue#200 ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement follow-up
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
war-and-code/dircue#201 ·
I maintainer di solito rispondono entro 1 giorno
-
follow-up
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
war-and-code/dircue#179 ·
I maintainer di solito rispondono entro 1 giorno
-
follow-up
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
war-and-code/dircue#178 ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement follow-up
Difficoltà 4/5 3-5 giorni Idoneità per principianti 45/100
war-and-code/dircue#172 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di war-and-code/dircue
Issue simili
-
raised-by:verifier
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
medici-finance/assay#2718 ·
I maintainer di solito rispondono entro 1 giorno
-
connector good first issue help wanted
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
soma-intelligence/open-health-sync#60 ·
I maintainer di solito rispondono entro 1 giorno
-
automation models
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
microsoft/TypeScript#64758 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
SocialGouv/iterion#2369 ·
I maintainer di solito rispondono entro 1 giorno