fix(cua-driver): visible Hyprland GTK button pixel clicks refuse despite working AT-SPI action
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- linux, rust
- Domain
- accessibility, desktop
Research direction
Start with platform-linux/src/tools/click.rs and trace the background left-click path into platform-linux/src/atspi/native.rs, especially perform_action_at_screen_point and its accessible-frame hit testing. Compare the reported window and button coordinates with the Wayland coordinate adapter, then run the visible GTK fixture reproduction with the optional input plugin absent. Done means a background click inside the enabled button increments its counter without changing focus, while an outside point does not invoke it.
Written by the indexing model from the issue text.
Description
Summary
On native Hyprland without the optional input plugin, background pixel clicks inside a visible GTK 3 button return background_unavailable, although Driver exposes the button's correct bounds and its AT-SPI click action works by element token. The Linux guide documents an AT-SPI-at-point route for this case. Four tested points inside the button refused; no raw-input backend is requested by this report.
Primary area
Cua Driver
Steps to reproduce
- Use the runtime below with the optional Hyprland input plugin absent. Start the release binary in the graphical session with an isolated home and its own socket,
CUA_DRIVER_RS_ENABLE_WAYLAND=1, telemetry disabled, andCUA_DRIVER_EXPERIMENTAL_HYPRLAND_INPUTandCUA_INJECT_SOCKETunset. The tested daemon usedserve --socket <scratch>/d.sock --no-overlay. - Save the fixture below as
fixture.pyin the scratch directory. It requires Python 3, PyGObject, and GTK 3. It creates a button and entry and writes the actual click count and entry text tofixture-state.jsonbeside itself. Run it through a silent-workspace compositor exec withenv -u NO_AT_BRIDGE GDK_BACKEND=wayland python3 <scratch>/fixture.pyso GTK publishes its accessibility tree. - Make that workspace visible on a temporary headless output while leaving the person's active window and workspace unchanged. In the executed run,
hyprctl output create headless TESTcreated an unfocused 1920×1080 output at scale 2 with workspace 21;hyprctl dispatch 'hl.dsp.exec_cmd("[workspace 21 silent] env -u NO_AT_BRIDGE GDK_BACKEND=wayland python3 <scratch>/fixture.py")'launched the fixture there. Select the unused workspace actually shown on the created output if it differs. - Use
call --socket <scratch>/d.sock list_windows '{}'to obtain the fixture's actualpidandwindow_id. Confirm it is native Wayland andis_on_screen:true. Read it withget_window_state, using a shared nonemptysession,tree_format:"elements", andscreenshot_out_file. Confirm the image shows the fixture and its button. - Call
clickwith that session, pid, window_id, and window-imagex:468,y:17, omitting element_token and delivery_mode. Those were the center coordinates in the observed 936×490 image; select the button center from the returned image when geometry differs. Readfixture-state.jsonto check actual delivery. - As a control, call
clickon the fresh button element_token instead of x/y. Read the counter again. Refresh the window snapshot before further attempts. - Close the fixture, stop only the isolated daemon, and remove only the temporary headless output with
hyprctl output remove TEST.
Save the script as described above; a counter increment means GTK received the button action, independently of Driver's result text.
Reproduction script: fixture.py
import gi, os, json
from pathlib import Path
gi.require_version("Gtk", "3.0")
from gi.repository import Gtk
base=Path(__file__).parent
w=Gtk.Window(title="W7 Driver Fixture")
w.set_default_size(600,200)
b=Gtk.Box(orientation=Gtk.Orientation.VERTICAL,spacing=8)
w.add(b)
button=Gtk.Button(label="Clicks: 0")
entry=Gtk.Entry()
entry.set_text("abc")
b.pack_start(button,False,False,0)
b.pack_start(entry,False,False,0)
count=0
def save():
(base/"fixture-state.json").write_text(json.dumps({"pid":os.getpid(),"clicks":count,"text":entry.get_text()}))
def click(*args):
global count
count+=1
button.set_label("Clicks: "+str(count))
save()
button.connect("clicked",click)
entry.connect("changed",lambda *args:save())
w.connect("destroy",Gtk.main_quit)
w.show_all()
save()
Gtk.main()
Expected behavior
The documented background pixel-left-click route should resolve an enabled actionable AT-SPI button under the point and invoke it without activating the window. The button counter should increment, as it does for the element-token control. If this particular supported surface cannot use that route, the limitation should be explicit rather than contradicting the documented AT-SPI fallback.
Actual behavior
list_windows reported is_on_screen:true. The 936×490 screenshot showed the button across local x=0..935, y=0..33. The tree exposed role:"button", actions:["click"], enabled:true, and global frame:{x:2572,y:38,w:936,h:34}, matching window_bounds:{x:2572,y:38,width:936,height:490}. The image and accessibility geometry therefore identify the same button.
All four background pixel attempts at (468,17), (100,17), (800,17), and (468,25) exited 1 with the following selected output. A new window read preceded each of the last three attempts. The first attempt left the counter at 0; the element-token control then incremented it to 1, and the remaining three pixel attempts left it at 1. The entry remained abc; the person's active window and active physical-output workspace did not change.
{
"code": "background_unavailable",
"detail": "the requested target has no focus-free input backend; the remaining input route can only deliver to the globally focused widget",
"escalation": {"recommended": "foreground"}
}
The successful token click returned exit 0, route:"accessibility", and effect:"unverifiable"; the fixture's counter supplied the independent confirmation. No cause is asserted for the pixel hit-test refusal.
Runtime or environment
- Executed binary: checksum-verified
cua-driver-rs-v0.52.3Linux x86_64 release archive;--versionreports Cua Driver 0.52.3. The canonical installer selected this release. - Host: Arch Linux, Hyprland 0.56.2, native Wayland GTK 3 fixture, no loaded plugins. The physical output remained at 3840×2160 scale 1.5; the temporary headless output was at global logical origin (2560,0), 1920×1080 scale 2.
- Current default branch inspected in a fresh clone:
mainat3f1661d00f382145720dea568ccde58b993fc0f0. The click routing, AT-SPI-at-point implementation, and Wayland coordinate adapter have no differences from the release tag. The default branch was not built or runtime-tested.
Evidence
- Expected source:
libs/cua-driver/rust/Skills/cua-driver/LINUX.md:317-323states that a pixel left click can deliver when hit-testing resolves to an actionable AT-SPI control. The shippeddescribe clickalso says background delivery first resolves the accessible under the point and fires its action. - Failure source: Executed CLI results, image/tree geometry, and fixture-owned counter values quoted above.
- Evidence provenance: observed
- Local verification: reproduced
- Reproduction completeness: complete
The current platform-linux/src/tools/click.rs:1704-1708 calls perform_action_at_screen_point for a background unmodified single left click; platform-linux/src/atspi/native.rs:5662-5738 reconstructs accessible frames and selects an action at that point. Both paths are relative to libs/cua-driver/rust/crates/. This identifies the intended route, not a proven causal diagnosis.
Restoration check
Repeat the visible-workspace fixture test without the plugin. A background pixel click inside the enabled button must increment its app-owned counter without changing the person's focus or workspace. Keep the token action as a control. Test a point outside the button separately so a passing result cannot come from invoking an arbitrary control regardless of coordinates.
Workaround and additional context
The verified workaround is to use the button's current element_token. That increments the counter through AT-SPI without taking focus. This report concerns the AT-SPI-at-point fallback, not adding raw background pointer input or requiring the optional plugin. The temporary output and fixture were removed and the original monitor geometry was restored after testing.
References
- https://github.com/trycua/cua/issues/1921 — closed historical native-Wayland click routing failure reported false success; this current GTK case returns an explicit refusal despite an actionable semantic target.
- https://github.com/trycua/cua/issues/3061 — closed coordinate-frame history; this case has a correct window image and matching button geometry but the background semantic pixel route refuses.
- https://github.com/trycua/cua/pull/5604 — related logical-coordinate work remains open; no claim is made that it fixes the observed AT-SPI-at-point failure.
Submission checks
- I searched for duplicate issues and active pull requests.
- This issue describes a single problem.
- I removed credentials, private data, sensitive screenshots, and vulnerability details from this public report.
Triage assessment
- Impact level: P2
- Assessment status: supported
- Impact basis: Four clicks inside one visible GTK button refused on the documented semantic pixel route. This blocks pixel-addressed use of that control, but the sample does not establish the scope across other apps or compositor configurations.
- Workaround status: available
- Workaround basis: Clicking the current element_token incremented the same fixture button's counter through the accessibility route without changing the person's focus or workspace.
- Dominant language
- Rust
- Stars
- 32.3k
- Forks
- 2.4k
- Avg merge
- 4h 47m
- Merged PRs (30d)
- 711
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from trycua/cua
-
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day
Similar issues
-
app-request
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
flatpark/flatpark#1423 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
helix-editor/termina#35 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 5 days
-
Add coverage reportsOpentype:ci
Difficulty 2/5 1-3 hours Newbie friendliness 68/100