Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

CLI worker pools default to cpus-1 each with no user-facing cap — index builds can saturate many-core machines

Aperta
  • Qualcosa non va?
#2,527 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
javascript, node.js
Ambito
cli, performance

Direzione di ricerca

The report names lib/dist/extraction/parse-pool.js, lib/dist/mcp/query-pool.js, lib/dist/resolution/resolver-pool.js, the README, and CLI help for index and init. Start by reading how those pools resolve their sizes and where configuration and verbose startup output are handled. Done means the existing environment knobs are documented and a user-facing thread limit bounds aggregate worker usage, with resolved pool sizes shown at startup.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Title: CLI worker pools default to cpus-1 each with no user-facing cap — index builds can saturate many-core machines

Type: enhancement (missing resource-control knob) + documentation gap

Environment

  • macOS, Apple Silicon (darwin-arm64), 18 logical CPUs, 47 GB RAM
  • @colbymchenry/codegraph v1.6.1, global npm install (~/.codegraph/versions/v1.6.1, node CLI)

Observed behavior

  • A co-installed VS Code extension's native engine saturated my whole machine during a full index (load average 38 on 18 cores, ~1700 %CPU in ps). I reported that to its own maintainers separately — but on inspecting the CLI's source I found the same unbounded-pool pattern here, and throttled proactively. I have not isolated a CLI-only runaway reproduction; this report is about the configuration gap, not a claim of a CLI crash.

Findings (from the installed v1.6.1 source)

Three independently-sized worker pools, each defaulting to cpus - 1:

  1. lib/dist/extraction/parse-pool.js — resolveParsePoolSize(envVal, cpuCount) returns Math.max(1, Math.min(cpuCount - 1, DEFAULT_PARSE_POOL_CAP)) → 17 parse workers on an 18-core machine.
  2. lib/dist/mcp/query-pool.js — resolvePoolSize(envVal, cpuCount) returns Math.max(1, Math.min(cpuCount - 1, MAX_POOL_SIZE)) → 17 query workers.
  3. lib/dist/resolution/resolver-pool.js — ResolverPool.resolvePoolSize sizes its own pool from os.availableParallelism().

Because these run concurrently during index / init, the pools can collectively occupy every core, leaving no headroom — the machine is unresponsive for the duration of the build.

Working but undiscoverable knobs — these exist in code but are not documented anywhere I could find:

Env var Effect Where
CODEGRAPH_PARSE_POOL_SIZE caps the parse pool extraction/parse-pool.js
CODEGRAPH_QUERY_POOL_SIZE caps the query pool (0 = disable, serve reads in-process) mcp/query-pool.js
CODEGRAPH_PARALLEL_RESOLVE_MIN minimum refs before the resolver pool engages resolution/resolver-pool.js
CODEGRAPH_NO_PARALLEL_RESOLVE=1 kill switch, sequential resolution resolution/resolver-pool.js

There is no equivalent in user-facing config: codegraph index --help and codegraph init --help expose no thread/worker option, and codegraph.json (project config) has no thread setting.

Requests

  1. Document the env knobs above in the README — they are the only cap that exists today, and I only found them by grepping lib/dist.
  2. Add a first-class limit — e.g. maxThreads in codegraph.json and/or a --threads flag on index/init/sync — applied across all pools together (parse + query + resolve + embedding), so one number bounds total engine CPU.
  3. Default to leaving headroom. cpus - 1 per pool is reasonable for one pool; with three pools running concurrently the sum can hit the full core count. Either size pools from a budget (total ≤ ½ or ¾ of cores) or cap the aggregate.
  4. --verbose already prints worker lifecycle/memory info — please also print the resolved pool sizes at startup so users can confirm the cap took effect.

Current workaround (verified working) — export CODEGRAPH_PARSE_POOL_SIZE=5 and CODEGRAPH_QUERY_POOL_SIZE=4 (rayon/tokio env vars do not apply to this Node CLI; those are only for the native Rust engine build).

Lingua principale
C
Stelle
73.2k
Fork
4.7k
Merge medio
2h 16m
PR unite (30g)
423

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di colbymchenry/codegraph

Tutte le issue di colbymchenry/codegraph

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.