Intermittent ReadFile stalls after guest idle — not reproduced, bisector included

Aperta
  • Qualcosa non va?
#73 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
28/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
go, grpc
Ambito
distributed-systems, observability

Direzione di ricerca

Review the capture tooling under .specs/tools/, especially wedge and e03probe, and use their documented probe modes to understand the existing measurements. A useful outcome would be a reproducible stall with receiver-side latency distributions or a clear pointer to existing guest-read instrumentation; the issue does not define a specific code change.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

question
Summary

On a self-hosted single-node k3s deployment I've observed intermittent multi-second (at times tens of seconds) ReadFile latency on paths that had been reading normally moments earlier. Occurrences correlate loosely with guest inactivity. Working hypothesis class: a wake-up path — suspended guest or idle connection state — blocking on the read. I have not been able to reproduce it under controlled conditions, so I'm filing this for visibility with capture tooling attached, rather than a speculative fix.

Investigation to date

Probes driven by a purpose-built bisector against a real router + guest:

Probe Result
Missing-path reads, cold and warm NotFound in ~25 ms, no stalls
Sequential read load (150 ops) clean
Forced suspend → immediate warm read guest auto-resumed; read completed in 185 ms
Composite (follow-stream + poller + mid-run suspend) 60/60 reads clean
Asks
  1. Has anyone observed multi-second ReadFile stalls correlated with guest idle or suspend/resume?
  2. Is there existing latency instrumentation on the guest read path (metrics or trace spans) that could confirm or refute wake-up attribution? If yes, pointers appreciated. If not, that instrumentation may be worth adding independently — it would turn the next occurrence into evidence rather than anecdote.
  3. If it recurs on our side, I'll attach receiver-side latency distributions captured with the tooling below.
Capture tooling

Available on fork branch barceloneta-k3s under .specs/tools/:

  • wedge — targeted bisector: segment-read patterns (s1/s2/s3), plus getenv, suspend, warm-hang, load, soak modes
  • e03probe — transcript-egress latency probe: follow-stream vs polling vs segmented reads, plus microbenchmarks (shell echo round-trip, StartProcess → first chunk, ReadFile at 4 KB / 256 KB / 1 MB)

Both emit JSON lines on stdout, so runs are directly diffable.

Environment
  • Single-node k3s; guests exercised both run-to-completion and under suspend/resume
  • Standard client → api → router → guest gRPC path
Lingua principale
Go
Stelle
71
Fork
17
Merge medio
21h 36m
PR unite (30g)
19

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di agent-substrate/env

Tutte le issue di agent-substrate/env

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.