Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

CLI worker pools default to cpus-1 each with no user-facing cap — index builds can saturate many-core machines

Abierto
  • ¿Algo no está bien?
#2,527 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
45/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
javascript, node.js
Área
cli, performance

Línea de trabajo

The report names lib/dist/extraction/parse-pool.js, lib/dist/mcp/query-pool.js, lib/dist/resolution/resolver-pool.js, the README, and CLI help for index and init. Start by reading how those pools resolve their sizes and where configuration and verbose startup output are handled. Done means the existing environment knobs are documented and a user-facing thread limit bounds aggregate worker usage, with resolved pool sizes shown at startup.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Title: CLI worker pools default to cpus-1 each with no user-facing cap — index builds can saturate many-core machines

Type: enhancement (missing resource-control knob) + documentation gap

Environment

  • macOS, Apple Silicon (darwin-arm64), 18 logical CPUs, 47 GB RAM
  • @colbymchenry/codegraph v1.6.1, global npm install (~/.codegraph/versions/v1.6.1, node CLI)

Observed behavior

  • A co-installed VS Code extension's native engine saturated my whole machine during a full index (load average 38 on 18 cores, ~1700 %CPU in ps). I reported that to its own maintainers separately — but on inspecting the CLI's source I found the same unbounded-pool pattern here, and throttled proactively. I have not isolated a CLI-only runaway reproduction; this report is about the configuration gap, not a claim of a CLI crash.

Findings (from the installed v1.6.1 source)

Three independently-sized worker pools, each defaulting to cpus - 1:

  1. lib/dist/extraction/parse-pool.js — resolveParsePoolSize(envVal, cpuCount) returns Math.max(1, Math.min(cpuCount - 1, DEFAULT_PARSE_POOL_CAP)) → 17 parse workers on an 18-core machine.
  2. lib/dist/mcp/query-pool.js — resolvePoolSize(envVal, cpuCount) returns Math.max(1, Math.min(cpuCount - 1, MAX_POOL_SIZE)) → 17 query workers.
  3. lib/dist/resolution/resolver-pool.js — ResolverPool.resolvePoolSize sizes its own pool from os.availableParallelism().

Because these run concurrently during index / init, the pools can collectively occupy every core, leaving no headroom — the machine is unresponsive for the duration of the build.

Working but undiscoverable knobs — these exist in code but are not documented anywhere I could find:

Env var Effect Where
CODEGRAPH_PARSE_POOL_SIZE caps the parse pool extraction/parse-pool.js
CODEGRAPH_QUERY_POOL_SIZE caps the query pool (0 = disable, serve reads in-process) mcp/query-pool.js
CODEGRAPH_PARALLEL_RESOLVE_MIN minimum refs before the resolver pool engages resolution/resolver-pool.js
CODEGRAPH_NO_PARALLEL_RESOLVE=1 kill switch, sequential resolution resolution/resolver-pool.js

There is no equivalent in user-facing config: codegraph index --help and codegraph init --help expose no thread/worker option, and codegraph.json (project config) has no thread setting.

Requests

  1. Document the env knobs above in the README — they are the only cap that exists today, and I only found them by grepping lib/dist.
  2. Add a first-class limit — e.g. maxThreads in codegraph.json and/or a --threads flag on index/init/sync — applied across all pools together (parse + query + resolve + embedding), so one number bounds total engine CPU.
  3. Default to leaving headroom. cpus - 1 per pool is reasonable for one pool; with three pools running concurrently the sum can hit the full core count. Either size pools from a budget (total ≤ ½ or ¾ of cores) or cap the aggregate.
  4. --verbose already prints worker lifecycle/memory info — please also print the resolved pool sizes at startup so users can confirm the cap took effect.

Current workaround (verified working) — export CODEGRAPH_PARSE_POOL_SIZE=5 and CODEGRAPH_QUERY_POOL_SIZE=4 (rayon/tokio env vars do not apply to this Node CLI; those are only for the native Rust engine build).

Lenguaje dominante
C
Estrellas
73.2k
Forks
4.7k
Merge medio
12 h 30 min
PR fusionados (30 d)
558

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de colbymchenry/codegraph

Todos los issues de colbymchenry/codegraph

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.