fix(cua-driver): visible Hyprland GTK button pixel clicks refuse despite working AT-SPI action

Open
#7,909 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
linux, rust

Research direction

Start with platform-linux/src/tools/click.rs and trace the background left-click path into platform-linux/src/atspi/native.rs, especially perform_action_at_screen_point and its accessible-frame hit testing. Compare the reported window and button coordinates with the Wayland coordinate adapter, then run the visible GTK fixture reproduction with the optional input plugin absent. Done means a background click inside the enabled button increments its counter without changing focus, while an outside point does not invoke it.

Written by the indexing model from the issue text.

Description

Linux

Summary

On native Hyprland without the optional input plugin, background pixel clicks inside a visible GTK 3 button return background_unavailable, although Driver exposes the button's correct bounds and its AT-SPI click action works by element token. The Linux guide documents an AT-SPI-at-point route for this case. Four tested points inside the button refused; no raw-input backend is requested by this report.

Primary area

Cua Driver

Steps to reproduce

  1. Use the runtime below with the optional Hyprland input plugin absent. Start the release binary in the graphical session with an isolated home and its own socket, CUA_DRIVER_RS_ENABLE_WAYLAND=1, telemetry disabled, and CUA_DRIVER_EXPERIMENTAL_HYPRLAND_INPUT and CUA_INJECT_SOCKET unset. The tested daemon used serve --socket <scratch>/d.sock --no-overlay.
  2. Save the fixture below as fixture.py in the scratch directory. It requires Python 3, PyGObject, and GTK 3. It creates a button and entry and writes the actual click count and entry text to fixture-state.json beside itself. Run it through a silent-workspace compositor exec with env -u NO_AT_BRIDGE GDK_BACKEND=wayland python3 <scratch>/fixture.py so GTK publishes its accessibility tree.
  3. Make that workspace visible on a temporary headless output while leaving the person's active window and workspace unchanged. In the executed run, hyprctl output create headless TEST created an unfocused 1920×1080 output at scale 2 with workspace 21; hyprctl dispatch 'hl.dsp.exec_cmd("[workspace 21 silent] env -u NO_AT_BRIDGE GDK_BACKEND=wayland python3 <scratch>/fixture.py")' launched the fixture there. Select the unused workspace actually shown on the created output if it differs.
  4. Use call --socket <scratch>/d.sock list_windows '{}' to obtain the fixture's actual pid and window_id. Confirm it is native Wayland and is_on_screen:true. Read it with get_window_state, using a shared nonempty session, tree_format:"elements", and screenshot_out_file. Confirm the image shows the fixture and its button.
  5. Call click with that session, pid, window_id, and window-image x:468,y:17, omitting element_token and delivery_mode. Those were the center coordinates in the observed 936×490 image; select the button center from the returned image when geometry differs. Read fixture-state.json to check actual delivery.
  6. As a control, call click on the fresh button element_token instead of x/y. Read the counter again. Refresh the window snapshot before further attempts.
  7. Close the fixture, stop only the isolated daemon, and remove only the temporary headless output with hyprctl output remove TEST.

Save the script as described above; a counter increment means GTK received the button action, independently of Driver's result text.

Reproduction script: fixture.py
import gi, os, json
from pathlib import Path
gi.require_version("Gtk", "3.0")
from gi.repository import Gtk
base=Path(__file__).parent
w=Gtk.Window(title="W7 Driver Fixture")
w.set_default_size(600,200)
b=Gtk.Box(orientation=Gtk.Orientation.VERTICAL,spacing=8)
w.add(b)
button=Gtk.Button(label="Clicks: 0")
entry=Gtk.Entry()
entry.set_text("abc")
b.pack_start(button,False,False,0)
b.pack_start(entry,False,False,0)
count=0
def save():
 (base/"fixture-state.json").write_text(json.dumps({"pid":os.getpid(),"clicks":count,"text":entry.get_text()}))
def click(*args):
 global count
 count+=1
 button.set_label("Clicks: "+str(count))
 save()
button.connect("clicked",click)
entry.connect("changed",lambda *args:save())
w.connect("destroy",Gtk.main_quit)
w.show_all()
save()
Gtk.main()

Expected behavior

The documented background pixel-left-click route should resolve an enabled actionable AT-SPI button under the point and invoke it without activating the window. The button counter should increment, as it does for the element-token control. If this particular supported surface cannot use that route, the limitation should be explicit rather than contradicting the documented AT-SPI fallback.

Actual behavior

list_windows reported is_on_screen:true. The 936×490 screenshot showed the button across local x=0..935, y=0..33. The tree exposed role:"button", actions:["click"], enabled:true, and global frame:{x:2572,y:38,w:936,h:34}, matching window_bounds:{x:2572,y:38,width:936,height:490}. The image and accessibility geometry therefore identify the same button.

All four background pixel attempts at (468,17), (100,17), (800,17), and (468,25) exited 1 with the following selected output. A new window read preceded each of the last three attempts. The first attempt left the counter at 0; the element-token control then incremented it to 1, and the remaining three pixel attempts left it at 1. The entry remained abc; the person's active window and active physical-output workspace did not change.

{
  "code": "background_unavailable",
  "detail": "the requested target has no focus-free input backend; the remaining input route can only deliver to the globally focused widget",
  "escalation": {"recommended": "foreground"}
}

The successful token click returned exit 0, route:"accessibility", and effect:"unverifiable"; the fixture's counter supplied the independent confirmation. No cause is asserted for the pixel hit-test refusal.

Runtime or environment

  • Executed binary: checksum-verified cua-driver-rs-v0.52.3 Linux x86_64 release archive; --version reports Cua Driver 0.52.3. The canonical installer selected this release.
  • Host: Arch Linux, Hyprland 0.56.2, native Wayland GTK 3 fixture, no loaded plugins. The physical output remained at 3840×2160 scale 1.5; the temporary headless output was at global logical origin (2560,0), 1920×1080 scale 2.
  • Current default branch inspected in a fresh clone: main at 3f1661d00f382145720dea568ccde58b993fc0f0. The click routing, AT-SPI-at-point implementation, and Wayland coordinate adapter have no differences from the release tag. The default branch was not built or runtime-tested.

Evidence

  • Expected source: libs/cua-driver/rust/Skills/cua-driver/LINUX.md:317-323 states that a pixel left click can deliver when hit-testing resolves to an actionable AT-SPI control. The shipped describe click also says background delivery first resolves the accessible under the point and fires its action.
  • Failure source: Executed CLI results, image/tree geometry, and fixture-owned counter values quoted above.
  • Evidence provenance: observed
  • Local verification: reproduced
  • Reproduction completeness: complete

The current platform-linux/src/tools/click.rs:1704-1708 calls perform_action_at_screen_point for a background unmodified single left click; platform-linux/src/atspi/native.rs:5662-5738 reconstructs accessible frames and selects an action at that point. Both paths are relative to libs/cua-driver/rust/crates/. This identifies the intended route, not a proven causal diagnosis.

Restoration check

Repeat the visible-workspace fixture test without the plugin. A background pixel click inside the enabled button must increment its app-owned counter without changing the person's focus or workspace. Keep the token action as a control. Test a point outside the button separately so a passing result cannot come from invoking an arbitrary control regardless of coordinates.

Workaround and additional context

The verified workaround is to use the button's current element_token. That increments the counter through AT-SPI without taking focus. This report concerns the AT-SPI-at-point fallback, not adding raw background pointer input or requiring the optional plugin. The temporary output and fixture were removed and the original monitor geometry was restored after testing.

References

Submission checks

  • I searched for duplicate issues and active pull requests.
  • This issue describes a single problem.
  • I removed credentials, private data, sensitive screenshots, and vulnerability details from this public report.

Triage assessment

  • Impact level: P2
  • Assessment status: supported
  • Impact basis: Four clicks inside one visible GTK button refused on the documented semantic pixel route. This blocks pixel-addressed use of that control, but the sample does not establish the scope across other apps or compositor configurations.
  • Workaround status: available
  • Workaround basis: Clicking the current element_token incremented the same fixture button's counter through the accessibility route without changing the person's focus or workspace.
Dominant language
Rust
Stars
32.3k
Forks
2.4k
Avg merge
4h 47m
Merged PRs (30d)
711

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from trycua/cua

All issues in trycua/cua

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.