nesquena/hermes-webui

bug(ux): mobile dictation stops on thinking pauses (SpeechRecognition not continuous) + no wake lock

Closed

#4,732 opened on Jun 22, 2026

 (3 comments) (0 reactions) (0 assignees)Python (2,316 forks)github user discovery
bughelp wantedsprint-candidateux

Repository metrics

Stars
 (16,946 stars)
PR merge metrics
 (Avg merge 14h 31m) (314 merged PRs in 30d)

Description

Summary

Browser-based speech-to-text (dictation) on mobile is unreliable in two ways:

  1. It stops transcribing too early — when the user pauses to think, the recognizer ends and the transcription stops, even though they're not done speaking. The tester attributed it to "VAD triggering early."
  2. The app doesn't stay awake the whole time, so listening is interrupted when the screen/app sleeps.

The tester notes this is mobile-specific — on desktop, STT "isn't VAD based and completely captures all speech until manually stopped." (On desktop, server-side STT via /api/transcribe records continuously until the user stops; on mobile the browser SpeechRecognition path is used and behaves differently.)

Where this lives

WebUI (nesquena/hermes-webui) — the voice/dictation logic in static/boot.js. Affects mobile web, PWA, and the native app wrappers (all use this same web-layer SpeechRecognition path). Wake-lock is a web API (navigator.wakeLock), so it belongs here too.

Root cause

1. Early stop — SpeechRecognition.continuous = false

The browser recognizer is configured to end after a single utterance:

static/boot.js:696-701:

function _ensureSpeechRecognition(){
  if(!SpeechRecognition) return null;
  const sr=recognition||new SpeechRecognition();
  sr.continuous=false;        // ← ends after one utterance / on first sustained pause
  sr.interimResults=true;
  ...

With continuous=false, the Web Speech API fires onend as soon as it detects an end-of-speech pause — exactly the "I paused for thought and it stopped" symptom. There's no auto-restart on onend (boot.js:717-731 just commits and optionally sends), so a natural thinking pause terminates the session. This is the browser's built-in endpointing, which the tester perceives as an over-eager VAD.

2. No wake lock — app sleeps mid-dictation

There is no Screen Wake Lock anywhere in the codebase (grep -rn "wakeLock" static/ → no matches). While dictating, nothing holds the screen awake, so on mobile the screen can dim/lock and the WebView gets frozen by the OS, cutting off recognition. Desktop doesn't hit this because the machine stays awake.

Suggested fix / fix shape

A. Keep listening across thinking pauses. Two options:

  • Set sr.continuous = true for the dictation (non-hands-free) path so the recognizer keeps going through pauses, OR

  • Keep continuous=false but auto-restart in onend while the user is still actively recording (i.e. they haven't tapped stop), accumulating into _finalText. The restart approach is more robust across engines that ignore continuous.

    Guard against the error/not-allowed path (don't auto-restart on a real permission/network error — boot.js:733-745), and stop cleanly when the user taps the mic or sends. Distinguish "user-initiated stop" from "engine endpointed" so we only auto-restart the latter.

B. Hold a wake lock while recording. Request navigator.wakeLock.request('screen') when recording starts (_setRecording(true) / mic start), release it on stop/onend/onerror and on visibilitychange (re-acquire on return-to-foreground). Lazy/optional — degrade silently if the API is unavailable (Wake Lock requires a secure context, consistent with the existing HTTPS dictation requirement from #4571).

C. (Optional) Consider routing mobile dictation through server STT (/api/transcribe, the MediaRecorder path) when a server STT provider is available — it records continuously until the user stops, matching the desktop behavior the tester prefers, and sidesteps the browser endpointing entirely. The plumbing already exists (_probeServerSttCapability, _forceMediaRecorder, boot.js:640-770); this would be a preference/default change rather than new infrastructure. Weigh against latency and the secure-context requirement.

Related

  • #4571 (closed) — dictation "mic access denied" on insecure HTTP origins. Different root cause (secure-context gate); this is about continuous capture + wake lock once dictation is working. Cross-referencing for the shared voice surface.

Scope

Medium, WebUI only — changes in static/boot.js voice block (continuous/auto-restart logic + wake-lock acquire/release). No server changes for A/B; C is a default/preference change reusing existing transcribe plumbing. A test around the auto-restart-on-endpoint vs stop-on-user-action distinction would lock it in.

Environment

  • Mobile (Android app / mobile web), browser SpeechRecognition dictation path.
  • Source: Hermes Discord (#webui), Android tester, 2026-06-22. Verified against WebUI master v0.51.591.
  • Labels: bug, ux.

Contributor guide