Windows voice to text leaves text in the prompt composer

Tracking

This report covers two submission paths, tracked as separate subissues:

• #1226 — Enter/IME handling, addressed by PR #1068.
• #1227 — Investigate the reported Send-button race during Windows Voice Typing.

Keep this parent issue open until both paths have been resolved or explicitly dispositioned.

Describe the bug
I am a heavy user of the voice typing that is part Windows. (Win+h). Under certain circumstances if you use Windows Voice typing to enter text in the prompt area If you hit the enter key or click the send button, the text remains in the prompt without being cleared.

To Reproduce
Steps to reproduce the behavior:

1. Go to https://www.openui.com/chat?form_factor=desktop
2. Click in the prompt area
3. Hit Win+h and dictate some text
4. Hit the enter key or click the send button

Note that the issue appears to be some sort of a race issue. You may need to hit enter pretty quickly.

Expected behavior
The prompt area should be clear.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Windows 11
• Browser Edge 151.0.4129.101, Chrome 151.0.7922.174

Additional context
I was able to resolve the issue by creating a custom composer which is mainly a copy of Composer.tsx with a few modifications clearly documented below.

// Windows Voice Typing (dictation) commits recognized speech into the field through an IME
// composition session that, per the report this fix is based on, stays open for as long as
// dictation is active — even across a real, physical Enter keypress typed by hand (not a spoken
// "Enter" command) once the user believes a phrase is done. The library's built-in composer
// (@openuidev/react-ui's AgentInterface/components/Composer.tsx) has no composition awareness at
// all: its onKeyDown treats *any* Enter as submit, including one that lands while
// `event.isComposing` is still true. So handleSubmit fires and clears the box on whatever text
// has streamed in so far, and then the still-open composition session finalizes a moment later
// and fires one more onChange with the (re-)committed dictated text, which repopulates the
// now-empty field — reproduced as "clears, then the spoken text reappears." (With DevTools open,
// event timing shifts enough that the race doesn't happen and everything looks fine — that was
// the first symptom reported.) assistant-ui's own composer (ComposerInput.js) guards against
// exactly this by bailing out of Enter-to-submit whenever `e.nativeEvent.isComposing` (or a
// compositionstart/end-tracked ref, since isComposing's timing relative to keydown isn't fully
// consistent across browsers) is true — the mirrored guard below. This is a full reimplementation
// via AgentInterface.Composer's documented "Mode C" (children replace the composer entirely)
// rather than a fix inside the library, since there's no prop-level hook into the built-in
// composer's internals. See FRAMEWORK_ISSUES.md's OpenUI section.
//
// Trade-off inherent to this fix, not a bug: while the composition session is still open, Enter
// no longer submits — it's left to the browser/IME to finalize first, same as any other IME-based
// input. If Voice Typing keeps composition open across the whole dictation session, the first
// physical Enter after dictating may just close out composition without sending, and a second
// Enter is needed to actually submit. That's standard behavior for IME input generally, not
// something to chase further — doing so is what caused the original race.
function OpenUiComposer() {
  const { processMessage, cancelMessage, isRunning, isLoadingMessages } = useThread();
  const [textContent, setTextContent] = useState("");
  const inputRef = useRef<HTMLTextAreaElement>(null);
  // FIX (not in upstream Composer.tsx): ref-tracked fallback for `e.nativeEvent.isComposing`,
  // read in onKeyDown below — see the file-level comment on why isComposing's timing isn't trusted alone.
  const isComposingRef = useRef(false);
  const [hasOverflowTop, setHasOverflowTop] = useState(false);
  const [hasOverflowBottom, setHasOverflowBottom] = useState(false);

  const updateOverflow = useCallback(() => {
    const input = inputRef.current;
    if (!input) return;
    const maxScrollTop = input.scrollHeight - input.clientHeight;
    setHasOverflowTop(maxScrollTop > 0 && input.scrollTop > 0);
    setHasOverflowBottom(maxScrollTop > 0 && input.scrollTop < maxScrollTop - 1);
  }, []);

  const handleSubmit = () => {
    if (!textContent.trim() || isRunning || isLoadingMessages) return;
    processMessage({ role: "user", content: textContent });
    setTextContent("");
    // FIX (not in upstream Composer.tsx): the Send button's onClick below calls handleSubmit
    // directly, with no composing check — unlike the Enter-key path, this one can still fire mid-
    // dictation. Forcing the DOM value empty (not just React state) narrows the same race for that
    // path: without it, e.currentTarget.value in onCompositionEnd could still read old-plus-newly-
    // committed text once the session finalizes. Best-effort only — setting .value mid-composition
    // isn't guaranteed to reset the IME consistently across browsers.
    if (inputRef.current) inputRef.current.value = "";
  };

  useLayoutEffect(() => {
    const input = inputRef.current;
    if (!input) return;
    input.style.height = "0px";
    input.style.height = `${Math.max(input.scrollHeight, 24)}px`;
    updateOverflow();
  }, [textContent, updateOverflow]);

  return (
    <div
      className="openui-agent-thread-composer"
      onClick={(e) => {
        if (!(e.target as HTMLElement).closest("button, a, [role='button']")) inputRef.current?.focus();
      }}
    >
      <div
        className="openui-agent-thread-composer__input-wrapper"
        data-overflow-top={hasOverflowTop || undefined}
        data-overflow-bottom={hasOverflowBottom || undefined}
      >
        <textarea
          ref={inputRef}
          value={textContent}
          autoFocus
          onChange={(e) => setTextContent(e.target.value)}
          onScroll={updateOverflow}
          // FIX (not in upstream Composer.tsx): upstream has no onCompositionStart/onCompositionEnd
          // handlers at all — these are the composition-tracking half of the guard.
          onCompositionStart={() => {
            isComposingRef.current = true;
          }}
          onCompositionEnd={(e) => {
            isComposingRef.current = false;
            // Syncs the IME's finalized text: compositionend can commit text without a preceding
            // onChange in some browsers, so relying on onChange alone would drop the last commit.
            setTextContent(e.currentTarget.value);
          }}
          className="openui-agent-thread-composer__input"
          placeholder="Type your query here"
          rows={1}
          onKeyDown={(e) => {
            if (e.key === "Enter" && !e.shiftKey) {
              // FIX (not in upstream Composer.tsx): upstream's onKeyDown goes straight to
              // preventDefault()+handleSubmit() here with no composition check at all — this bails
              // out instead, leaving Enter to the browser/IME while dictation is still composing.
              if (e.nativeEvent.isComposing || isComposingRef.current) return;
              e.preventDefault();
              handleSubmit();
            }
          }}
        />
        <div className="openui-agent-thread-composer__action-bar">
          <IconButton
            onClick={isRunning ? cancelMessage : handleSubmit}
            icon={isRunning ? <Square size="1em" fill="currentColor" /> : <ArrowUp size="1em" />}
            size="extra-small"
            variant="primary"
            aria-label={isRunning ? "Cancel message" : "Send message"}
            className="openui-agent-thread-composer__submit-button"
          />
        </div>
      </div>
    </div>
  );
}