cua-driver Linux X11: set_value keyboard fallback types into the window behind a scrolled-off field and reports ok

Open
#7,945 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
56/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
linux, rust

Research direction

Start with libs/cua-driver/rust/crates/platform-linux/src/tools/set_value.rs, focusing on resolve_element_local_coords, the with_x11_foreground body, and set_value_result. Compare the fallback's bounds and focus checks with the click tool's point_ownership hit test, then inspect the existing result and read-back handling. Done means clicks outside the target or without focus send no keys, and an empty read-back reports an error.

Written by the indexing model from the issue text.

Description

Problem

On Linux X11, the foreground keyboard fallback of set_value can type into another application's window and still report the step as ok.

The fallback (path=click_type_fg, platform-linux/src/tools/set_value.rs) clicks the centre of the element's cached snapshot bounds with an XTest click, presses Ctrl+A, types the value and presses Tab. It does not check that the click point is inside the target window's visible area or that the click put keyboard focus on that element. When the field has scrolled out of its window's viewport since the snapshot, the cached centre falls outside the window. The real pointer click then lands on whatever window lies under that screen point, and the typed text goes there.

Seen with an Electron chat window whose blocker form sits below the fold, over a LibreOffice Calc window. Five set_value calls on the form's fields each returned:

set_value ok: Set value of element [29] to 'LHP-C17' (path=click_type_fg); but the element reads back '' — the write may not have committed.

The values went into the Calc window behind it: its Controls sheet gained the five values in cells below its data. The form then reported "Please fill out this field." The agent retried after scrolling, and the second attempt landed. The stray cell writes stayed in the workbook, which was saved later in the run.

This reproduces on current releases: the same sequence left extra cells in the Calc sheet on every run of a five-run check.

Expected

  • Before the XTest click, the fallback checks that the element's centre lies inside the target window's visible bounds and that the topmost window at that point belongs to the target (the click tool's point_ownership hit test already does this for element clicks). Otherwise it refuses with element_bounds_unavailable, or scrolls the element into view first.
  • After the click, the fallback checks that keyboard focus is in the target element before it types. If it is not, it sends no keys.
  • An empty read-back after the write is returned as a failure (effect: "suspected_noop", is_error), not as ok, so a batch stops.

Where

libs/cua-driver/rust/crates/platform-linux/src/tools/set_value.rs, the keyboard fallback (resolve_element_local_coords and the with_x11_foreground body), and set_value_result.

Dominant language
Rust
Stars
32.3k
Forks
2.4k
Avg merge
4h 47m
Merged PRs (30d)
711

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from trycua/cua

All issues in trycua/cua

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.