Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Streaming ASR leaks previous-turn punctuation into the next final (leading ? / ,)

クローズ
  • 何か問題がありますか？
#22 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 2 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
45/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
cpp, grpc
領域
audio-video-rtc, backend

調査の方向性

riva_server の streaming endpointing/finalization と punctuation のパスから始め、persistent Riva gRPC stream と記載された endpointing flags を使用して、遅延する句読点を再現します。完了条件は、末尾の句読点が直前の utterance の final に含まれ、punctuation-only final や次の hypothesis の先頭の句読点が発生しないことです。

索引モデルが issue の本文から書いたものです。

説明

Summary

With streaming ASR endpointing enabled on Nemotron Speech Streaming 0.6B, trailing punctuation from one utterance is not always attached to that utterance's is_final result. It arrives later as its own final, or as a leading ? / , / . on the next utterance.

Example from a live gRPC session (riva_server in nvcr.io/nvidia/nemo-speech.cpp:0.1.0):

Transcription: [Hi, can you tell me what is the capital of UK]
(user turn ends)
Transcription: [ ? What is the capital]
Transcription: [ of United States of America]

The first utterance is missing the question mark. The next utterance starts with that leftover ?.

This is not a client transcript-sort issue. The pipeline concatenates is_final chunks in receive order. The punctuation is missing from the earlier final and present on the later one.

Repro

  1. Serve ASR with endpointing on Nemotron Speech Streaming 0.6B (--asr.endpointing.enable=true, --asr.endpointing.stop_history_eou_ms=400).
  2. Keep one streaming Recognize stream open across multiple spoken turns (Riva gRPC, as a voice-agent client does).
  3. Ask a question that should end with ?, wait for the bot, then ask another question.
  4. Repeat a few turns.

Also seen with a leading comma:

Transcription: [ , okay]
Transcription: [ ? And what is the capital]

Expected

Punctuation that belongs to utterance N is included in that utterance's final, or flushed before the next speech segment starts. The next final must not start with leftover ?, ,, or . from the previous turn.

Actual

The word text finalizes first without trailing punctuation. After the next user turn has already started, a later final carries that punctuation. Sometimes it is punctuation-only ([ ?]). Sometimes it is prepended to the next hypothesis ([ ? What is the capital]). A punctuation-only final can also start a new user turn on the client (TranscriptionUserTurnStartStrategy).

Observed timing (one session, UTC):

  • 07:10:56.827 [ London] (no ?)

  • 07:10:56.829 user stopped

  • 07:11:00.482 next user started

  • 07:11:05.343 [ ?]

  • 07:12:19.572 [Hi, can you tell me what is the capital of UK] (no ?)

  • 07:12:19.573 user stopped

  • 07:12:23.288 next user started

  • 07:12:24.853 [ ? What is the capital]

Root cause

Streaming endpointing is supposed to emit one final per utterance and reset decoder state so delayed tokens do not leak. Trailing punctuation is still emitted after that reset, on the same Recognize stream.

Likely the punctuation path (self-punctuation on Nemotron RNNT, or a delayed rewrite of . / , / ?) is not flushed with the EOU final. It is applied when more audio arrives, so it lands on the next segment.

The client cannot tell the leftover ? apart from real next-turn text, because both arrive as is_final on the same stream.

Environment

  • nvcr.io/nvidia/nemo-speech.cpp:0.1.0
  • Nemotron Speech Streaming EN 0.6B GGUF (nemotron-speech-streaming-en-0.6b.q8_0.gguf)
  • Endpointing on, stop_history_eou_ms=400
  • Persistent Riva gRPC streaming Recognize (Pipecat NvidiaSTTService)
  • No separate --pnc-model passed. Punctuation is coming from the ASR path itself.

Request

Flush trailing punctuation with the utterance that produced it. Do not emit a later punctuation-only final, and do not prepend previous-turn ? / , / . onto the next hypothesis. Tracking this here so GGML riva_server and the streaming ASR client stay aligned.

主要言語
C++
スター
167
フォーク
37
平均マージ
6日 23時間
マージ済み PR（30日）
9

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA/NeMo-Speech.cpp のほかの issue

NVIDIA/NeMo-Speech.cpp の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。