Measure and reduce newgit's power draw and compute cost

Aperta
  • Qualcosa non va?
#324 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
git, linux, macos, rust
Ambito
performance, tooling

Direzione di ricerca

Start by reading the command implementations and polling paths named in the issue, especially crates/newgit-core/src/compose.rs, supervisor.rs, lock.rs, fresh_base.rs, manager.rs, and source.rs. Then identify a realistic agent session to benchmark and record wall time, CPU time, subprocess counts, I/O, disk use, and idle cost; the issue also calls for remote CPU measurements. Done means a repeatable benchmark has run, baseline numbers are recorded, and each confirmed hotspot is fixed with before and after numbers or filed separately.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

performance

newgit runs underneath agents that call it constantly, often many instances at once on a laptop. Whatever it costs per command gets multiplied by every agent, every instance, and every turn. Right now we don't know what that cost is. This issue is about measuring it first and then cutting what turns out to be waste.

Why it matters

  • Agents call newgit far more often than people call git. A 50 ms overhead that nobody notices by hand becomes real CPU time and battery drain across a fleet of parallel instances.
  • Idle cost is worse than per-command cost. Anything that keeps waking up (supervised processes, polling, retries) drains power even when nobody is working.
  • On the remote, CPU time costs money. Worker CPU time is billed, and the remote now pauses itself when a week's bill passes its budget (#314). Wasted work there means hitting that pause sooner.

Leads worth checking (unmeasured, not confirmed problems)

Process and I/O cost per command

  • How many git subprocesses each command spawns (spawn, checkpoint, undo, status, run, capture, merge). Fork and exec on every call adds up, so it's worth checking whether a single in-process read (gix/git2) would serve the read-only paths.
  • The compose/split engine (crates/newgit-core/src/compose.rs): what it costs per commit, and whether that cost grows with the number of bound trackers or with history length.
  • Workspace materialization: each workspace is a full independent clone. Measure the time, disk I/O, and bytes written per spawn, and check whether CoW cloning (APFS clonefile, reflinks) is actually being used wherever it could be (store, installs, tracker projection).
  • status: does it rescan the whole workspace every time? Agents run it in loops.

Wakeups and polling

  • Fixed-interval sleep loops: supervisor.rs (20/100/300 ms), lock.rs (10/200 ms), fresh_base.rs (50 ms), manager.rs (200 ms readiness), source.rs. Each is fine on its own. The question is whether any of them run for long, or while idle, and whether an event (pidfd/kqueue, fs notify, a lock wait) could replace the poll.
  • Does anything newgit starts keep running between commands, and what does it cost while idle?
  • Port readiness probes (TcpStream::connect_timeout loops).

Remote (remote/)

  • CPU ms per request for the hot paths: the Git endpoint (clone/fetch), checkpoint upload, and auth.
  • Repeated work that could be cached, such as recomputing pack or ref listings and re-verifying tokens.

Proposed approach

  1. Measure first, no optimizing yet. Add a small benchmark script under scripts/ that runs a realistic agent session (spawn N instances, edit, checkpoint, undo, status loop, merge, drop) and records:
    • wall time and CPU time (user and sys) per command
    • number of subprocesses spawned per command
    • bytes read and written, and disk used per instance
    • idle CPU wakeups / energy impact with N instances alive (powermetrics / top -stats power on macOS, perf stat / powertop on Linux)
    • remote: CPU ms per request from Worker analytics
  2. Write down the baseline (a short docs/ note or a comment here) so later changes can be compared against it.
  3. Fix the biggest items, smallest change first. Each fix should come with a before/after number.
  4. Consider a CI check for regressions on subprocess count and CPU time for a couple of core commands, along the lines of #183 for web pages.

Done when

  • A repeatable benchmark exists and has been run on macOS (and ideally Linux).
  • We have baseline numbers for per-command cost, idle cost with N instances, and remote CPU per request.
  • Every confirmed hot spot is either fixed with before/after numbers or filed as its own issue.
  • Idle instances use roughly no CPU, or the docs explain why they can't.
Lingua principale
Rust
Stelle
0
Fork
0
Merge medio
7h 58m
PR unite (30g)
97

Preparare l'ambiente

Non abbiamo ancora controllato i file di configurazione di questo progetto. Parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Spheroman/newgit

Tutte le issue di Spheroman/newgit

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.