Report the retry budget in assertion failure messages

未关闭
  • 哪里不对？
#5,223 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
3/5
预计耗时
1-2 天
新手友好度
72/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
playwright, python
领域
testing

调研方向

从 Browser/assertion_engine.py 开始，结合现有的 DEBUG 统计信息阅读 with_assertion_polling 及其两个 raise 位置。检查 atest/test/08_Scope_Tests/scope_keywords.resource 和仓库的验收测试，以评估消息的兼容性和放置位置。完成的标准是：重试后的断言失败会暴露重试预算，而首次尝试失败仍保留现有消息。

由索引模型根据 Issue 内容生成。

描述

priority: medium

Is your feature request related to a problem? Please describe.

When an assertion keyword fails, the failure message says nothing about the fact
that the assertion was retried, or for how long. Get Element States id=not_here contains hidden
fails with:

Elements states '['detached']' (list) should contain '['hidden']' (list)

That message is identical whether the assertion failed instantly or was retried for a
full second before giving up. There is no hint that retrying happened, that it is
configurable, or that Set Retry Assertions For exists.

This is a discoverability problem more than a debugging one. Users who do not already
know about retry_assertions_for have no path to discovering it, because the one moment
they would care about it — an assertion that failed after quietly retrying — is exactly
the moment the library stays silent. Anecdotally, experienced Robot Framework users are
surprised the setting exists at all.

The information is already computed. Browser/assertion_engine.py logs it at DEBUG in the
finally block of with_assertion_polling:

Assertion polling statistics:
First element asserted in: 0.2534 seconds
Total tries: 4
Elapsed time in retries 0.7864 seconds

DEBUG is the wrong level for something a user needs in order to know a feature exists.

Describe the solution you'd like

Include the retry budget in the assertion failure message itself, at the two raise sites
in with_assertion_polling (Browser/assertion_engine.py).

Gate it on retries having actually happened (tries > 1). That is both the only case where
the information is useful and the thing that keeps the change small: an assertion that failed
on its first attempt, or one running with retry_assertions_for=0, keeps its message
byte-for-byte.

Describe alternatives you've considered

  • Leave it at DEBUG. Status quo. Does not solve discoverability — nobody turns on DEBUG
    before they know there is something to look for.
  • Document it harder. Does not reach the user at the moment they need it.
  • Always append, ungated. Larger compatibility surface for no extra benefit, since a
    non-retried assertion has no retry budget worth reporting.

Additional context

Compatibility. Appending to assertion messages is a breaking change for
Run Keyword And Expect Error patterns that match exactly. Measured in this repository's
own acceptance tests:

Run Keyword And Expect Error sites 225
pattern uses a glob / regexp 141
exact match 84

The tries > 1 gate narrows the affected set considerably — the exact-match sites are
mostly non-retrying errors (Error: Crash, plugin errors) rather than assertion-keyword
failures. The residual should be measured before implementing. The same consideration
applies to users' suites, which is why this needs release notes.

Design question — where the information goes. The blast radius depends on placement, and
that is worth deciding deliberately rather than defaulting to "append to the end":

  • appended as a trailing sentence — simplest, but touches the message that exact matches see;
  • a separate WARN/INFO log line beside the failure — zero message compatibility cost,
    but easier for a user to miss;
  • attached only when the assertion exhausted its budget (as opposed to any retry at all).

Secondary benefit. retry_assertions_for is currently the one scope setting whose effect
cannot be verified except with a stopwatch. atest/test/08_Scope_Tests measures wall-clock
duration to infer it, and that is the source of roughly 20 CI failures across the last ~1200
runs. Every other scope setting can be verified from a message — Playwright names the timeout
it used (Timeout 1500ms exceeded), and strict mode raises strict mode violation. Naming
the retry budget in the failure would let those tests assert on a message too, and drop the
last timing-based assertion in that suite.

Related: Browser/assertion_engine.py, atest/test/08_Scope_Tests/scope_keywords.resource.

主要语言
Python
星标
662
派生
149
平均合并
6 小时 9 分钟
30 天内合并 PR
119

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

MarketSquare/robotframework-browser 的其他 Issue

查看 MarketSquare/robotframework-browser 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。