Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

One-shot Pyodide executions retain a high sidecar memory watermark after VM disposal

Aperta
  • Qualcosa non va?
#1,902 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
javascript, python, rust, wasm
Ambito
backend, performance, testing-qa

Direzione di ricerca

Inizia leggendo crates/execution/src/python.rs, crates/execution/src/javascript.rs, assets/runners/python-runner.mjs e crates/native-sidecar/tests/python.rs. Esegui la sonda dedicated-sidecar con esecuzioni ripetute della libreria standard, di NumPy e di pandas, registrando PSS/RSS e le metriche del ciclo di vita. Il lavoro è completato quando viene stabilito se la memoria raggiunge un plateau limitato oppure viene definito un limite di recupero supportato e documentata la strategia di produzione.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

Repeated one-shot Python executions substantially raise the shared native sidecar's proportional set size (PSS). Waiting for each execution, disposing its VM, deleting its persisted workspace, and waiting five seconds does not return the sidecar close to its cold memory footprint. Exiting the sidecar process does.

A fresh-process staged probe suggests a retained/high-watermark behavior rather than enough evidence for an unbounded linear leak:

  • cold sidecar: ~18 MiB PSS
  • standard-library Python, after 5 isolated executions: ~270 MiB
  • NumPy, after 8 additional isolated executions: ~422 MiB
  • pandas, across 8 additional isolated executions: ~445–558 MiB, non-monotonic

The pandas stage sometimes drops by ~80 MiB, so this issue intentionally does not claim that every execution permanently leaks a fixed amount. The operational problem is that VM disposal is not currently a reliable physical-memory reclamation boundary for one-shot Pyodide workloads in a long-lived shared sidecar.

Environment

  • @rivet-dev/agentos: 0.2.16-rc.1
  • Node.js: v22.22.2
  • OS: Linux x86_64, kernel 6.6.98
  • glibc: 2.38
  • sidecar placement: shared process
  • Python mode: one-shot vm.python.spawn, not retained contexts
  • packages: Pyodide-provided NumPy and pandas assets

Probe procedure

The probe starts a fresh Runtime/native sidecar and records Linux PSS from /proc/<sidecar-pid>/smaps_rollup.

For every iteration it:

  1. creates a new VM/workspace;
  2. calls vm.python.spawn(...);
  3. waits for the execution to finish;
  4. disposes the VM;
  5. deletes the workspace SQLite files;
  6. waits 5 seconds;
  7. records sidecar PSS.

Workloads are executed sequentially in this order:

cold baseline
standard library x5
NumPy x8
pandas x8

Each execution uses a distinct VM. There are no retained Python contexts.

Results

PSS values are KiB:

baseline  0   18,374

stdlib    1  241,806
stdlib    2  263,672
stdlib    3  269,970
stdlib    4  270,846
stdlib    5  276,930

numpy     1  356,644
numpy     2  372,698
numpy     3  397,170
numpy     4  412,674
numpy     5  413,298
numpy     6  416,974
numpy     7  420,218
numpy     8  421,774

pandas    1  558,232
pandas    2  527,151
pandas    3  444,790
pandas    4  461,963
pandas    5  476,763
pandas    6  486,619
pandas    7  539,143
pandas    8  461,629

Execution times were approximately 19–23 seconds for standard-library/NumPy runs and 26–30 seconds for pandas runs.

A separate already-warm probe started around 971 MiB PSS and reached approximately 1,162 MiB after three NumPy+pandas executions, despite deleting each workspace and waiting after each run. Restarting the Runtime/sidecar returned it to approximately 31 MiB.

File descriptors returned to the baseline range, and deleting SQLite/workspace files did not lower PSS. The memory belongs to the native sidecar process rather than the Node host process.

Expected behavior / requested contract

Returning exactly to cold PSS after every execution may be unrealistic for V8, Pyodide, WebAssembly memories, and the native allocator. What is needed for a long-lived shared sidecar is one of:

  1. repeated identical one-shot workloads reach a documented, bounded plateau after VM/context disposal; or
  2. AgentOS exposes a supported reclamation/lifecycle boundary that releases Python runtime memory without terminating unrelated VMs in the shared sidecar.

At minimum, it would be useful to document the expected memory lifecycle and recommended production strategy.

Suspected boundary

Python runs inside the shared sidecar approximately as:

native sidecar
  -> V8 session
    -> Pyodide
      -> WebAssembly linear memory / ArrayBuffer backing stores
        -> NumPy / pandas

AgentOS has context/VM disposal paths and removes logical tracking state, but logical disposal does not demonstrate that V8 sessions, WASM backing stores, or allocator arenas are released or returned to Linux.

Possible causes include:

  • disposed V8/Pyodide objects still being referenced;
  • delayed collection of WASM/ArrayBuffer backing stores;
  • native allocator arenas retaining released pages;
  • a combination of these.

The non-monotonic pandas measurements make allocator/runtime high-watermark retention the leading explanation, but a longer 30–100 iteration low-point slope is needed to rule out an unbounded component.

Potentially relevant source areas:

  • crates/execution/src/python.rs
  • crates/execution/src/javascript.rs
  • crates/execution/assets/runners/python-runner.mjs
  • crates/native-sidecar/tests/python.rs

The Python integration tests also note shared-process teardown/init instability for V8/Pyodide-backed cases and process-global Pyodide module state, which may be relevant to defining the supported lifecycle.

Suggested regression / diagnostics

A useful upstream regression would run in a dedicated sidecar OS process:

cold start
standard library x10
NumPy x30
pandas x30

For every iteration:

  • wait for one-shot execution completion;
  • dispose context and VM;
  • wait a fixed settle period;
  • record PSS/RSS/private dirty, active VM/process/context counts, and V8/WASM memory metrics.

After discarding package materialization/warm-up iterations, evaluate the slope of local PSS minima rather than requiring memory to return to cold baseline. Heap/allocation profiling could then distinguish reachable V8 sessions, retained WASM backing stores, Rust/JS tracking objects, and allocator-only retention.

Why this matters

A shared sidecar can serve many independent VMs. If the reliable physical reclamation boundary is the sidecar process rather than the VM, operators need to drain and restart the entire sidecar based on memory thresholds, affecting unrelated sessions. A bounded per-VM or Python-runtime reclamation boundary would avoid that operational coupling.

Lingua principale
Rust
Stelle
4.7k
Fork
263
Merge medio
8h 57m
PR unite (30g)
30

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di rivet-dev/agentos

Tutte le issue di rivet-dev/agentos

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.