Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Recognize native build systems (Kbuild/Kconfig, CMake, Meson, Bazel/Buck, autotools, Make) as static component evidence

Aperta
#102 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cmake, go
Ambito
build-system

Direzione di ricerca

Start with the proposed new pkg/buildsys/ package and read the existing manifest-recognition implementation referenced as #5 to understand how adapters represent components and evidence. The first listed task is the Kbuild/Kconfig adapter; its acceptance cases are outlined in pkg/buildsys/kbuild_test.go, including object types, conditional symbols, descent, and comment handling. Done means the adapter and its tests meet those cases; the issue also describes a much broader set of adapters, fixtures, and integration work.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement

User outcome

  • The map of a C, C++ or systems codebase shows its real structure: build targets, libraries, executables, kernel subsystems and modules, and configuration options, with evidence.
  • Today such a map shows nothing, because those projects have no package-manager manifest.

Why

  • The kernel. Measured on the #94 candidate: the Linux kernel map had one component (tools/net/ynl, a small Python tool) and zero deployables. The kernel is the first repository many skeptical readers will try, and "one Python component" reads as broken.
  • The gap is structural. Component recognition today covers npm, Go, Python/uv, Cargo, Maven, Gradle and .NET. Those are ecosystems with declarative manifests. Most systems code declares its structure in build files instead: Kbuild/Kconfig, CMake, Meson, Bazel/Buck, autotools, plain Makefiles.
  • Static parsing is enough. These can be read as data for a large useful subset without evaluating arbitrary build logic, just as #5 does for manifests. For Bazel/Buck (Starlark) there's a mature parse-only Go library: bazelbuild/buildtools build package. That leverages existing work rather than re-implementing it.

Proposal

Static, bounded adapters that produce components, membership and local-dependency edges, each with declared_config evidence and honest partial coverage wherever evaluation would be needed:

  • Kbuild/Kconfig.
    • obj-y/obj-m/obj-$(CONFIG_X) lines produce built-in vs module objects per directory, and subdirectory descent (obj-y += dir/) produces containment.
    • config/menuconfig symbols are recorded with their defining file.
    • Conditional objects keep their controlling CONFIG_* symbol as a condition. Nothing is evaluated.
    • Subsystem components are derived from the directory structure as declared by Kbuild descent, not from guesswork.
  • CMake.
    • project(), add_executable, add_library (with STATIC/SHARED/INTERFACE), add_subdirectory and literal target_link_libraries produce targets and edges.
    • Variables, generator expressions, include() of computed paths and functions are recorded as unresolved references.
  • Meson. project, executable, library/shared_library/static_library, subdir and dependency() names.
  • Bazel/Buck(2). Parse BUILD and BUCK files with buildtools. Rule calls with literal name, srcs and deps produce targets and local edges; macros and load()ed rules are recorded as unresolved.
  • Autotools and Make.
    • configure.ac AC_INIT produces project identity; Makefile.am bin_PROGRAMS/lib_LTLIBRARIES produce targets.
    • Plain Makefiles contribute only explicit, literal target names from rules. Everything else is unknown, deliberately conservative.
  • Cross-checks (see the #94 finding on overclaimed complete): a directory with C/C++ sources and build files, but no recognized target, degrades the components question with unrecognized_build_declarations.

Tasks

  • Kbuild/Kconfig adapter first (the highest-visibility demo), including a scalable directory-descent graph for about 3,000 Makefiles.
  • CMake adapter (a literal subset, with an unresolved-reference taxonomy).
  • Bazel/Buck adapter via buildtools.
  • Meson and autotools adapters.
  • Plain-Make literal targets.
  • Summary rendering for systems repos: top subsystems by size and number of targets, not a flat list of 3,000 directories.

Test plan

Unit tests

New package pkg/buildsys/ (new) with one _test.go file per adapter:

  • pkg/buildsys/kbuild_test.go (new): table-driven cases for obj-y, obj-m, obj-$(CONFIG_X) lines (conditional object records its CONFIG_* symbol, not a resolved value); subdirectory descent via obj-y += dir/; config/menuconfig symbol extraction; skipping #-comment lines; a directory with only unrecognized assignment forms produces zero objects and zero subsystems; a Makefile with 3,000 entries parses in under 500 ms (time assertion).
  • pkg/buildsys/cmake_test.go (new): project(), add_executable, add_library with STATIC/SHARED/INTERFACE variants, add_subdirectory, target_link_libraries with literal names; a variable reference (${VAR}) in a target name produces an unresolved-reference record, not a target; a include() of a computed path is recorded as an unresolved include; generator expressions ($<TARGET_FILE:t>) are rejected as unresolved; if()/else() blocks are not evaluated — all branches contribute targets as partial candidates with conditional flag.
  • pkg/buildsys/bazel_test.go (new): cc_binary, cc_library, go_binary, py_library rule calls with literal name, srcs and deps; a load() that introduces a macro is recorded as an unresolved rule origin; a glob() in srcs is recorded as an unresolved source set; deps containing //external labels produce cross-workspace edges, which are dropped with an external_dep reason.
  • pkg/buildsys/meson_test.go (new): project(), executable(), library(), shared_library(), static_library(), subdir(), dependency() with a literal name; a dependency() with a computed name is unresolved.
  • pkg/buildsys/autotools_test.go (new): AC_INIT from configure.ac produces project identity; bin_PROGRAMS, lib_LTLIBRARIES from Makefile.am produce targets; a Makefile.am with only SUBDIRS produces subdirectory containment edges.
  • pkg/buildsys/make_test.go (new): explicit literal target names from top-level rules; phony targets (.PHONY) are excluded from components; pattern rules (e.g., %.o: %.c) are not reported as targets; an included Makefile at a computed path is recorded as an unresolved include.
  • pkg/buildsys/cross_test.go (new): a C directory with .c files and no recognized build file degrades to unrecognized_build_declarations coverage on the components question.
Hand-labeled ground truth and fixtures (tied to #75)

Ground truth is hand-labeled once and committed as JSON; any deviation blocks the PR.

Linux kernel (Kbuild/Kconfig):

  • New tests/build_adapters/labels/kernel/ (new). Labels cover:
    • the 20 top-level subsystem directories (arch/, drivers/, fs/, net/, mm/, kernel/, sound/, security/, crypto/, lib/, block/, ipc/, init/, firmware/, Documentation/, tools/, scripts/, include/, samples/, usr/);
    • a hand-labeled sample of 50 directories' obj-y and obj-m sets, drawn from drivers/usb/, net/ipv4/, fs/ext4/, arch/x86/kernel/, security/selinux/ (representative of conditional and unconditional objects);
    • 10 config/menuconfig symbol definitions from Kconfig files at arch/x86/Kconfig, drivers/Kconfig, net/Kconfig.
  • Precision/recall gate: ≥ 95% precision and ≥ 90% recall on the labeled obj-y/obj-m sets; 100% recall on top-level subsystem containment edges.
  • The kernel source tree is not committed; tests/build_adapters/fetch_repos.py (new) fetches a pinned tag (v6.10) under a .gitignored cache directory (per #88).

CMake (abseil-cpp or the CMake project itself):

  • Labels in tests/build_adapters/labels/cmake/ (new): top-level project() identity, all add_library and add_executable targets at depth ≤ 2, and 10 target_link_libraries edges with literal deps. Pinned commit.
  • Fetched by tests/build_adapters/fetch_repos.py, not committed.

Bazel (abseil-cpp with Bazel BUILD files):

  • Labels in tests/build_adapters/labels/bazel/ (new): cc_library targets from absl/strings/BUILD.bazel and absl/base/BUILD.bazel, plus deps edges that are literal local labels. Pinned commit.

Meson (systemd or libpng):

  • Labels in tests/build_adapters/labels/meson/ (new): top-level project(), first-level subdir() calls, executable and library targets with literal names. Pinned tag.

Autotools (GNU coreutils):

  • Labels in tests/build_adapters/labels/autotools/ (new): AC_INIT identity, bin_PROGRAMS targets from the top-level Makefile.am. Pinned tag.
Adversarial fixtures

Committed under tests/build_adapters/fixtures/adversarial/ (new). Each is a minimal synthetic directory tree, no binaries, generated inline as strings in a gen_adversarial.py (new) script:

  • macro-heavy CMake: 50 include() calls with computed paths, no literal targets → all unresolved, no crash.
  • generated Makefiles: a Makefile that includes a pattern-generated file → unresolved include, zero targets.
  • recursive Kbuild: a chain of 200 obj-y += dir/ descents → bounded at the configured depth limit, partial coverage.
  • large Kconfig: 5,000 config symbols in a single file → parses without OOM, counts reported accurately.
  • non-UTF-8 source file: a CMakeLists.txt with a Latin-1 encoded comment → parser skips the bad bytes, reports unresolved, no panic.
  • duplicate targets: two add_library(foo ...) calls → second is flagged as a duplicate with a diagnostic, first is retained.
Differential oracles (tied to #84)

Hermetic containers (new Dockerfiles in tests/build_adapters/docker/, --network=none at runtime), run on workflow_dispatch and on PRs touching pkg/buildsys/:

  • CMake file API (dircue-cmake-oracle:cmake-3.30, new): runs cmake -S . -B /tmp/build -DCMAKE_EXPORT_COMPILE_COMMANDS=ON followed by reading .cmake/api/v1/reply/ to extract targets; dircue's literal-subset target list must be a subset of the file-API list (precision gate ≥ 95%); recall is expected to be lower (macros and generator expressions produce API targets dircue can't see, which is correct behavior). The oracle never executes user build code — it reads only the file-API JSON reply, which CMake itself writes.
    • Oracle images are pinned (tag cmake-3.30); the Dockerfile is the source of truth for the version.
    • Labeled repo: abseil-cpp at the pinned commit.
  • Bazel query (dircue-bazel-oracle:bazel-7.4, new): bazel query //... --output=label (in a sandbox with --noenable_workspace); dircue's literal target list must be a subset of the query output. Same precision/recall framing.
    • Labeled repo: abseil-cpp at the pinned commit.
    • Note: Bazel downloads dependencies during query even with --noenable_workspace; the container image pre-fetches dependencies at build time so the runtime container has --network=none.
  • Kbuild cross-check (no container needed): compare dircue's obj-y/obj-m set for the labeled 50 directories against the hand labels; this is purely a label-match check, not a process-based oracle, since Kbuild can only be fully evaluated by the kernel build system.
Fuzz targets (tied to #87)
  • New pkg/buildsys/fuzz_test.go (new): FuzzKbuildLine(f *testing.F), FuzzCMakeLine(f *testing.F), FuzzBazelFile(f *testing.F), FuzzMesonFile(f *testing.F). Seeds: one small real file per system from the labeled repos. Properties: never panics, output is deterministic for identical input, byte count of unresolved references is non-negative.
  • Persisted corpora per #87; not in CI by default.
Mutation-testing focus (tied to #86)
  • The Kbuild obj-y / obj-m discriminator: mutation of the string comparison silently conflates built-in and module objects. Every labeled directory has at least one obj-m entry to catch this.
  • The CMake conditional-flag logic: mutation that drops the conditional flag causes phantom unconditional targets to be reported.
  • The tree-entry sort order in the Bazel deps edge list: dircue's output must be deterministic; mutation of sort direction changes the output but may not fail without a determinism assertion.
Performance and resource checks (tied to #89/#90)
  • Deterministic counters (new): buildsys_files_parsed, buildsys_targets_found, buildsys_unresolved_refs added to the map summary for any repo with recognized build files; gated non-negative in PR CI for the labeled repos.
  • Kernel Kbuild parse time: make bench-kbuild (new make target) runs pkg/buildsys/BenchmarkKbuild over a 3,000-Makefile synthetic tree; assert completes in under 30 s.
  • On-demand benchmark make bench-buildsys (new) runs all adapters over the labeled repos and reports targets/s and files/s.
E2E script
  • tests/build_adapters/e2e.py (new). For each labeled repo:
    1. Runs dircue map and extracts the component list and dependency edges.
    2. Compares against the hand labels: precision and recall per adapter.
    3. Where a container oracle exists, runs it and compares against oracle output.
    4. Logs to build_adapters_results.jsonl:
      • fields: repo, adapter, files_parsed, targets_found, unresolved_refs, label_precision, label_recall, oracle_precision, oracle_recall, wall_ms, rss_kb.
    5. Prints a scoreboard table with a pass/fail per adapter.
    6. Exits non-zero if any adapter's label precision is below 90% or recall below 80%.
CI placement
  • Unit tests (pkg/buildsys/...) run in the existing test job on every PR (fast, no network, no container).
  • Adversarial fixture tests run in the existing test job on every PR (deterministic, small).
  • Oracle containers (tests/build_adapters/e2e.py with container steps) run on workflow_dispatch and on PRs touching pkg/buildsys/; new job buildsys-oracle in .github/workflows/ci.yml (new job), --network=none, 15-minute timeout.
  • Kernel-scale Kbuild benchmark is workflow_dispatch / make bench-kbuild only (large fetch).
Regression fences
  • Any PR that reduces label_precision below 90% or label_recall below 80% on a previously passing adapter is a regression blocker.
  • A directory with recognized C/C++ sources and build files that produces zero targets must emit the unrecognized_build_declarations diagnostic; a grep-based check in the preflight CI job (extend existing) enforces this.
  • No build tool is ever invoked by dircue; a strace/dtrace check in the oracle container (run dircue under strace -e execve) asserts zero execve calls to cmake, make, bazel, meson, or ninja.

Risks

  • Overreach into evaluation. Keep a strict literal subset, and make "unresolved" first-class.
  • Kernel noise. The kernel has an enormous number of conditional objects. The summary needs aggregation to stay readable.

Success criteria

  • The kernel map shows its subsystems and module structure with evidence.
  • Measured precision and recall on labeled systems repos are published.
  • No build tool is ever executed by dircue.

Related: #5, #74, #75, #54, #23. Related idea issues: #103, #107.

Sequencing: Independent; do Kbuild first. Measured accuracy is published through #107.


This proposal came out of an idea-wizard session on 2026-09-23. It isn't a release commitment, and the milestone is deliberately unset so the owner can decide per idea. It must preserve dircue's contracts:

  • deterministic, offline output, with no execution of inspected content;
  • the legacy Linguist-compatible CLI and output unchanged;
  • unknown never rendered as absent;
  • no provider findings or verdicts imported;
  • CI event-driven only (no schedule:).
Lingua principale
Go
Stelle
0
Fork
0
Merge medio
5h 8m
PR unite (30g)
54

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di war-and-code/dircue

Tutte le issue di war-and-code/dircue

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.