Measure and reduce newgit's power draw and compute cost

Abierto
  • ¿Algo no está bien?
#324 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
git, linux, macos, rust
Área
performance, tooling

Línea de trabajo

Start by reading the command implementations and polling paths named in the issue, especially crates/newgit-core/src/compose.rs, supervisor.rs, lock.rs, fresh_base.rs, manager.rs, and source.rs. Then identify a realistic agent session to benchmark and record wall time, CPU time, subprocess counts, I/O, disk use, and idle cost; the issue also calls for remote CPU measurements. Done means a repeatable benchmark has run, baseline numbers are recorded, and each confirmed hotspot is fixed with before and after numbers or filed separately.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

performance

newgit runs underneath agents that call it constantly, often many instances at once on a laptop. Whatever it costs per command gets multiplied by every agent, every instance, and every turn. Right now we don't know what that cost is. This issue is about measuring it first and then cutting what turns out to be waste.

Why it matters

  • Agents call newgit far more often than people call git. A 50 ms overhead that nobody notices by hand becomes real CPU time and battery drain across a fleet of parallel instances.
  • Idle cost is worse than per-command cost. Anything that keeps waking up (supervised processes, polling, retries) drains power even when nobody is working.
  • On the remote, CPU time costs money. Worker CPU time is billed, and the remote now pauses itself when a week's bill passes its budget (#314). Wasted work there means hitting that pause sooner.

Leads worth checking (unmeasured, not confirmed problems)

Process and I/O cost per command

  • How many git subprocesses each command spawns (spawn, checkpoint, undo, status, run, capture, merge). Fork and exec on every call adds up, so it's worth checking whether a single in-process read (gix/git2) would serve the read-only paths.
  • The compose/split engine (crates/newgit-core/src/compose.rs): what it costs per commit, and whether that cost grows with the number of bound trackers or with history length.
  • Workspace materialization: each workspace is a full independent clone. Measure the time, disk I/O, and bytes written per spawn, and check whether CoW cloning (APFS clonefile, reflinks) is actually being used wherever it could be (store, installs, tracker projection).
  • status: does it rescan the whole workspace every time? Agents run it in loops.

Wakeups and polling

  • Fixed-interval sleep loops: supervisor.rs (20/100/300 ms), lock.rs (10/200 ms), fresh_base.rs (50 ms), manager.rs (200 ms readiness), source.rs. Each is fine on its own. The question is whether any of them run for long, or while idle, and whether an event (pidfd/kqueue, fs notify, a lock wait) could replace the poll.
  • Does anything newgit starts keep running between commands, and what does it cost while idle?
  • Port readiness probes (TcpStream::connect_timeout loops).

Remote (remote/)

  • CPU ms per request for the hot paths: the Git endpoint (clone/fetch), checkpoint upload, and auth.
  • Repeated work that could be cached, such as recomputing pack or ref listings and re-verifying tokens.

Proposed approach

  1. Measure first, no optimizing yet. Add a small benchmark script under scripts/ that runs a realistic agent session (spawn N instances, edit, checkpoint, undo, status loop, merge, drop) and records:
    • wall time and CPU time (user and sys) per command
    • number of subprocesses spawned per command
    • bytes read and written, and disk used per instance
    • idle CPU wakeups / energy impact with N instances alive (powermetrics / top -stats power on macOS, perf stat / powertop on Linux)
    • remote: CPU ms per request from Worker analytics
  2. Write down the baseline (a short docs/ note or a comment here) so later changes can be compared against it.
  3. Fix the biggest items, smallest change first. Each fix should come with a before/after number.
  4. Consider a CI check for regressions on subprocess count and CPU time for a couple of core commands, along the lines of #183 for web pages.

Done when

  • A repeatable benchmark exists and has been run on macOS (and ideally Linux).
  • We have baseline numbers for per-command cost, idle cost with N instances, and remote CPU per request.
  • Every confirmed hot spot is either fixed with before/after numbers or filed as its own issue.
  • Idle instances use roughly no CPU, or the docs explain why they can't.
Lenguaje dominante
Rust
Estrellas
0
Forks
0
Merge medio
6 h 21 min
PR fusionados (30 d)
173

Preparar el entorno

Aún no hemos revisado los archivos de configuración de este proyecto. Empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de Spheroman/newgit

Todos los issues de Spheroman/newgit

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.