Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

SpeakingRateStream blocks the event loop: per-frame Python STFT loop in `_stft` / `_spectral_flux`

Open
  • Is something wrong?
#7,902 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@omerekin is already working on this.

Since Oct 17, 2026.

  • #7903 by @omerekin — open

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python
Domain
audio-video-rtc, performance

Research direction

Locate SpeakingRateStream._stft and _spectral_flux, then trace audio draining through _main_task. Run the existing speaking-rate tests and event-loop timing checks, comparing spectral-flux output and loop-blocking measurements before and after the change. Done means the calculation is vectorized without changing results and queued TTS audio no longer causes the measured stalls.

Written by the indexing model from the issue text.

Description

TranscriptSynchronizer always runs SpeakingRateDetector on the agent's TTS audio, on the job's event loop. SpeakingRateStream._stft computes the STFT one frame at a time in a Python loop (about 80 np.fft.rfft calls per 1 s window at 24 kHz), and _spectral_flux loops over the frames again. Because RoomIO's _ParticipantAudioOutput.capture_frame queues TTS audio without pacing, _main_task drains a whole lump of a reply (10 windows per second of audio) without yielding.

Measured on livekit-agents 1.8.2, numpy 2.5.2, Apple M3 (the code is unchanged in 1.8.8, 2.0.0b1 and main):

  • 0.61 ms per window; _main_task draining 20 s of queued 24 kHz audio holds the loop ~123 ms (2 s: 7.6 ms, 5 s: 26 ms, 10 s: 58 ms).
  • In a local test conversation driven by synthetic speech, with LIVEKIT_AGENTS_LOOP_BLOCK_WARN_MS=50: 4-5 loop blocks of 59-131 ms per run, a few seconds into each agent turn. With the 200 ms output buffer, stalls that size can be audible, and on smaller cloud machines they'd be roughly twice as long.

The same computation vectorized gives a bit-identical result (480 random cases across 6 sample rates and 4 lengths) and is ~3.8x faster (0.16 ms per window; 20 s of queued audio holds the loop ~38 ms). It replaces _stft + _spectral_flux:

def _spectral_flux(
    self, audio: np.ndarray[tuple[int], np.dtype[np.float32]], sample_rate: int
) -> float:
    frame_length = int(sample_rate * 0.025)  # 25ms
    hop_length = frame_length // 2  # 50% overlap
    num_frames = (len(audio) - frame_length) // hop_length + 1
    if num_frames < 2:
        return 0.0
    window = np.hanning(frame_length)
    scale_factor = 1.0 / np.sqrt(np.sum(window**2))
    frames = np.lib.stride_tricks.sliding_window_view(audio, frame_length)[::hop_length]
    magnitudes = np.abs(np.fft.rfft(frames * window, axis=1) * scale_factor)
    return float(np.mean(np.sum(np.abs(np.diff(magnitudes, axis=0)), axis=1)))

It might also be worth yielding (await asyncio.sleep(0)) between windows in _main_task, so a long lump never holds the loop, whatever the per-window cost.

Happy to open a PR with this if it's useful.

Dominant language
Python
Stars
14.6k
Forks
3.9k
Avg merge
3d 7h
Merged PRs (30d)
165

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from livekit/agents

All issues in livekit/agents

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.