Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Livekit inference timeouts since 14th july

Aperta
  • Qualcosa non va?
#6,675 6 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Tranquilla
Stack tecnologico
python
Ambito
backend-api-design

Direzione di ricerca

Start with the supplied FallbackAdapter configuration and the linked LiveKit session records; compare the LiveKit Inference path with the SDK custom-provider path described in the issue. Investigate where the intermittent timeouts occur and whether they correlate with the model or region. Done means the cause is identified and the reported timeout behavior is addressed.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug
Bug Description

We noticed a significant increase in GPT-5.4-mini timeouts when using LiveKit Inference past few weeks.

We’ve been running GPT-5.4-mini in production for our real-time voice agents, and it has been very stable until recently. Starting from 14th July, we began seeing a noticeable increase in timeout errors, with 21 July being the worst day, where we recorded around 70 timeout incidents in a single day.

The issue appears to be intermittent, most requests succeed, but some requests stall and eventually time out, impacting the voice agent experience.

My traffic is dropped now because of low season, so timeouts too, but can be seen from this graph about pattern, on some days, i think weekends it dropped to minimal even when usage was similar and week days it went high up again.

I am 100% sure, there is no error on our side in configurations as nothing was changed since a long time. and it happens in mid of session once or twice and sometimes thrice as per recorded timeouts .

Expected Behavior

There should be zero to minimal timeouts.

Reproduction Steps
# currently primary is gemma4 and secondary is openai5.4 mini
# false means livekit inference and true means sdk custom using api key)

_DEFAULTS: dict[str, str] = {
    "openai": "gpt-4o-mini",
    "google": "gemini-3.1-flash-lite",
}

def _make_openai(cfg: AiConfigLlm, custom: bool = False) -> llm.LLM:
    extra: dict = {}
    if not custom:
        return inference.LLM(
            model=f"openai/{cfg.model or _DEFAULTS['openai']}",
            extra_kwargs=extra,
        )

    # code for custom is after it


def _make_google(cfg: AiConfigLlm, custom: bool = False) -> llm.LLM:
    if not custom:
        extra: dict = {}
        return inference.LLM(
            model=f"google/{cfg.model or _DEFAULTS['google']}",
            extra_kwargs=extra,
        )
    # code for custom is after it


primary = _make_google(
     AiConfigLlm(provider="google", model="gemma-4-31b-it"),
      custom=False
)
fallback = _make_openai(
     AiConfigLlm(provider="openai", model="gpt-5.4-mini"),
     custom=False
) 
adapters = [primary, fallback]

# LLM_ATTEMPT_TIMEOUT used was 2.5
return FallbackAdapter(adapters, attempt_timeout=settings.LLM_ATTEMPT_TIMEOUT)



But can be seen here in those sessions 
https://cloud.livekit.io/projects/p_310ab3c567u/sessions/RM_L5PMbNUoCjSH
https://cloud.livekit.io/projects/p_310ab3c567u/sessions/RM_uchq4ygwDQd8
Operating System

livekit cloud

Models Used

openai 5.4 mini gemma4 via livekit inference

Package Versions
"livekit-agents>=1.6.7",
"livekit-api>=1.2.0",

Deployed in livekit cloud
Session/Room/Call IDs

https://cloud.livekit.io/projects/p_310ab3c567u/sessions/RM_L5PMbNUoCjSH
https://cloud.livekit.io/projects/p_310ab3c567u/sessions/RM_uchq4ygwDQd8

Proposed Solution
Additional Context

To add on, it happens with recent released gemma4 model too, so i doubt its something with livekit inference, region we used was eu-central. If i use third party provider via sdk then its fine.

Full discussion here, but was not helpful and was not resolved
https://community.livekit.io/t/livekit-inference-timeouts-happening-since-past-2-weeks/1718/12

Screenshots and Recordings
Image
Lingua principale
Python
Stelle
14.6k
Fork
3.9k
Merge medio
2g 9h
PR unite (30g)
159

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di livekit/agents

Tutte le issue di livekit/agents

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.