Android: an identifier-only leaf paints its sibling layer over the whole screen, marking everything covered

未关闭 适合新手
  • 哪里不对？
#4,249 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
79/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
android, react-native, typescript
领域
mobile, testing

调研方向

从 packages/capture-kit/src/snapshot/android-replacement-surface-occlusion.ts 开始，追踪 semanticPaintRect 和 semanticShowRect 如何处理只有标识符的叶节点。Issue 中包含一个建议条件，并解释了为什么标识符应继续可用作目标，同时不计作已绘制区域。检查 Android 快照或遮挡测试（如果有），并验证兄弟按钮不再被标记为被覆盖，而标识符视图仍可被标记为被覆盖。

由索引模型根据 Issue 内容生成。

描述

Version: 0.21.36 (the same code is on main at packages/capture-kit/src/snapshot/android-replacement-surface-occlusion.ts)
Platform: Android (emulator, android-36), React Native app

What happens

On Android every element on our chat screen comes back interactionBlocked: "covered", so locator taps refuse with "Ref @e88 is covered by another visible element and cannot be tapped safely". The screen itself is fine: a coordinate click on the same button opens it.

Why

The screen is a React Native stack of full-screen sibling layers under one parent: background, message list, header buttons, composer. The composer layer is

#117 ViewGroup            (0,0,1080,2150)  not hittable
  #119 ViewGroup id=coach-container (0,0,1080,2150)  leaf, not hittable, no label/value
  #120 ... composer buttons (hittable)

coach-container is a <View testID pointerEvents="box-none" style={absoluteFill}>: transparent, no handlers, and it has only an identifier. In semanticPaintRect the clause childCount === 0 && hasSemanticContent(node) counts it as painting its whole rect, because hasSemanticContent accepts identifier. Its layer has an action (the composer buttons), so the layer's paint footprint covers 100% of every earlier sibling, and the header buttons and every message bubble get marked covered.

Suggested fix

Don't let an identifier-only leaf paint. An identifier is a test hook, not something drawn, and a non-hittable leaf with no label or value carries no evidence that it's opaque or catches touches:

function semanticPaintRect(node: RawSnapshotNode, childCount: number): Rect | undefined {
  if (!isPositiveFiniteRect(node.rect)) return undefined;
  return node.hittable === true ||
    isScrollableType(node.type) ||
    (childCount === 0 && Boolean(node.label?.trim() || node.value?.trim()))
    ? node.rect
    : undefined;
}

semanticShowRect keeps hasSemanticContent, so such a view can still be the target that gets covered. We're carrying this as a pnpm patch on 0.21.36 for now. I'll comment here with the result once our Android coach flows have run with it.

主要语言
TypeScript
星标
5.3k
派生
377
平均合并
14 小时 6 分钟
30 天内合并 PR
650

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

callstack/agent-device 的其他 Issue

查看 callstack/agent-device 的全部 Issue

相似的 Issue

更多 TypeScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。