Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

realtime WS: commit never yields .completed for utterances >= ~1s (0.5s works)

Aperta
  • Qualcosa non va?
#49 1 commento 1 reazione 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
52/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
audio-video-rtc, backend

Direzione di ricerca

Start with server/http/http_server.cpp, especially the realtime commit handler and the streaming finish path. Reproduce the issue with the raw WebSocket client using utterances from 0.5 to 2.0 seconds, then inspect how longer streams and endpointing handle finalization. Done means longer utterances emit committed and completed events, with endpointing no longer stalling.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

The realtime WebSocket transcription path in nemo-speech serve 0.1.0 never emits .completed (or input_audio_buffer.committed) after input_audio_buffer.commit once the streamed utterance reaches ~1 second or more. At 0.5 s it works (+0.19 s finalization); at 1.0 s, 1.5 s, and 2.0 s it hangs forever. Short-utterance success shows the protocol handling is correct, so this looks length-dependent in the streaming finish path.

A second, possibly related symptom: with --asr.endpointing.enable=true and 2 s of trailing zero-PCM silence, no endpoint .completed ever fires and audio_processed stalls partway through the stream.

Environment

  • Binary: nemo-speech 0.1.0 release archive nemo-speech-0.1.0-linux-x86_64-cuda.tar.gz
  • OS/GPU: NixOS 26.11, NVIDIA T1000 4 GB, driver 595.91, CUDA backend (backend=CUDA0)
  • Model: nemotron-speech-streaming-en-0.6b.q8_0.gguf (665 MiB)
  • Serve: nemo-speech serve --asr-model <gguf> --asr.batching.state_arena_slots 1 --host 127.0.0.1 --port 8080
  • Note: default arena slots OOM (cudaMalloc failed: out of memory) with ~2 GB already resident on the 4 GB card; state_arena_slots 1 was required to run at all.

Repro

Raw WebSocket client (stdlib only, masked client frames), repeated per trial on a fresh connection:

  1. GET /v1/realtime handshake, receive session.created.
  2. Send {"type": "session.update", "session": {"sample_rate": 16000, "language": "en-US"}}, receive session.updated.
  3. Stream 16 kHz mono PCM16 as binary frames in 100 ms (3200-byte) chunks, back-to-back.
  4. Send {"type": "input_audio_buffer.commit"}.
  5. Read events.

Audio: 16 kHz mono command utterances; trials used 0.5 s / 1.0 s / 1.5 s / 2.0 s prefixes of the same clip.

Observed

Streamed audio Result after commit
0.5 s .completed in +0.19 s, transcript "What?" (correct for the prefix)
1.0 s no committed, no .completed, no error (25 s wait)
1.5 s same hang
2.0 s same hang

Mid-stream .delta events flow correctly in all cases (full transcript visible within ~0.3 s of arrival, ~6x realtime), so ingestion and decoding work; only finalization never surfaces. Endpointing variant (--asr.endpointing.enable=true --asr.endpointing.stop_history_eou_ms=800, utterance + 2 s trailing silence): audio_processed advances to ~2.0-2.2 s of a 2.66 s stream then stalls; no endpoint final ever arrives.

Additional observation: streaming tail words are unstable across runs on identical input ("Play some Kendrick Lam" vs "Play some Kendrick"), and trailing silence does not flush the remainder.

Notes

  • Current main's server/http/http_server.cpp commit handler (ensure_stream(); emit(stream->finish()); ...committed) is identical to 0.1.0, so this likely reproduces on main (not built here; no toolchain on the test host).
  • Possibly related to #48 (streaming RNNT stall) and #40/#41 (EOU handling), but this repro uses endpointing off, no silence mid-stream, and fails deterministically on utterance length alone.
Lingua principale
C++
Stelle
150
Fork
32
Merge medio
7g 17h
PR unite (30g)
8

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/NeMo-Speech.cpp

Tutte le issue di NVIDIA/NeMo-Speech.cpp

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.