Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Azure STT: pronunciation assessment on the streaming recognizer

Abierto
  • ¿Algo no está bien?
#8,050 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
55/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
azure, python
Área
audio-video-rtc, backend

Línea de trabajo

Start by reading the Azure STT implementation in stt.py and its unit tests; the issue says the recognizer setup and _on_recognized are the relevant entry points. Check how SpeechData.metadata is populated for final transcripts, then compare the fork implementation linked in the issue. Done means an optional pronunciation assessment config adds Azure’s scores to each final transcript while leaving default behavior unchanged.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Feature Type

Would make my life easier

Feature Description

We're building a Japanese learning app on top of LiveKit Agents and we score the learner's pronunciation with Azure's pronunciation assessment.

Right now the Azure plugin doesn't let you touch the recognizer before it starts, and _on_recognized only keeps the text. So to get scores we have to send the same audio to Azure a second time, which means paying for the transcription twice.

Azure supports pronunciation assessment directly on continuous recognition: you just apply a PronunciationAssessmentConfig to the recognizer, and every final result comes back with the scores in its JSON.

What I'd like:

  • an optional pronunciation_assessment: speechsdk.PronunciationAssessmentConfig on azure.STT
  • the assessment attached to each final transcript, in SpeechData.metadata["pronunciation_assessment"] (the field is already there for plugin specific data)

Off by default, so nothing changes for anyone else.

Workarounds / Alternatives
  • Second request to Azure with the recorded audio after each turn: works, but double billing and extra latency.
  • Patching the plugin's private methods (_on_recognized, recognizer creation): works too, but it breaks on any plugin update, so we had to pin livekit-plugins-azure.
  • Forking the whole STT in our codebase: that's what we have now, I'd rather not maintain it.
Additional Context

I already have a small implementation on a fork (~40 lines in stt.py + unit tests), tested against the real Azure service with Japanese audio, scores come back on every final:
https://github.com/Bilnaa/agents/tree/azure-recognizer-hooks

Happy to open the PR if the approach looks ok to you, or change the shape of it (e.g. a generic recognizer setup callback instead).

Lenguaje dominante
Python
Estrellas
14.6k
Forks
3.9k
Merge medio
3 d 20 h
PR fusionados (30 d)
193

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de livekit/agents

Todos los issues de livekit/agents

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.