Voice input and read-aloud for chat

Aperta
#229 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
tauri, typescript

Direzione di ricerca

Start by running the UI in mock mode and reading apps/desktop/ui/src/views/ChatView.tsx and its composer and message action components. The issue asks for a design note before implementation; first agree on privacy defaults and platform support, then take one of the proposed slices as a separate PR. Done means meeting that slice's criteria, including the listed tests and platform verification.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

accessibility area: desktop-ui enhancement help wanted needs design

size: M · first-timer: no

Please comment before starting so we can agree on the design. Voice involves the microphone and possibly sending audio to a service, so the privacy defaults must be agreed first.

Summary

Voice input and read-aloud for chat: a push-to-talk microphone button in the composer that turns speech into text in the message box (never auto-sent), and a "read aloud" action on assistant messages.

Why it matters

Talking to a bot is natural on a desktop that is also running the bot, and read-aloud helps people who cannot or do not want to read long answers, so it is also an accessibility feature. Praxi is local-first, so the interesting part is doing this without silently shipping audio to a third party.

Current state

  • The UI has no speech input or output: there is no use of speech synthesis, speech recognition or microphone capture anywhere in apps/desktop/ui/src.
  • The composer and message actions live in apps/desktop/ui/src/views/ChatView.tsx and components/; the desktop shell is Tauri (apps/desktop/src/), so the web view's capabilities differ per OS (Windows, Linux and macOS web views do not all expose the same speech APIs, and a browser's built-in recognition may send audio to the browser vendor's servers).
  • Providers are configurable (providers/catalog/*.toml, OpenAI-compatible endpoints), some of which offer an audio transcription endpoint; Ollama and LM Studio run locally.

Proposal and suggested approach

Settle these in a design note: what runs locally by default, what needs an explicit opt-in because audio leaves the machine (and a clear one-time notice naming where it goes), where the microphone permission is requested, and what happens on platforms where the web view lacks the API.

Slices (each its own PR):

  1. Read aloud (smallest, no audio leaves the machine): a speaker button on assistant messages using the OS voices through speech synthesis, stop/pause control, language chosen from the message or the UI locale, and a Settings toggle. Never reads a message that is still streaming or tool output.
  2. Dictation, local-first. Use a platform recognizer only where it is documented to run on the device; otherwise a configured provider's transcription endpoint as an opt-in with the notice above. The transcript goes into the composer for the user to edit and send; it never sends by itself.
  3. Settings for both, English and Turkish strings, keyboard shortcut and accessible labels.

Acceptance criteria

  • Design note agreed; the default install sends no audio anywhere.
  • Read aloud works for English and Turkish messages and stops when the chat changes or the window closes.
  • Dictation fills the composer, never submits, and shows a clear state (listening, processing, error); no recording is stored on disk.
  • Unit tests for the state machines; check:i18n, lint, typecheck and UI tests pass; verified manually on at least one OS, with the others documented as untested or unsupported.

Out of scope

Always-on wake-word listening, voice conversations with streamed audio, voice cloning, and text-to-speech through paid cloud services.

How to get started

Run the UI in mock mode (pnpm --dir apps/desktop/ui dev, open http://localhost:1420/?mock=1), read the composer and message actions in views/ChatView.tsx, and check which speech APIs your OS web view exposes. Comment here with the platform you would target first.

Lingua principale
Rust
Stelle
0
Fork
1
Merge medio
6h 54m
PR unite (30g)
152

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di emreocell/Praxi-Bot

Tutte le issue di emreocell/Praxi-Bot

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.