Flaky: profile-hover smoke spec derives its expected value from a mid-transition sample

Open Beginner friendly
#6,237 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
90/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
playwright, typescript

Research direction

Start with desktop/tests/e2e/message-feedback-snapshots.spec.ts:97-114 and inspect the existing waitForAnimations helper and the profile-hover smoke spec. Run that E2E spec, then verify the sampled channel color is settled and repeated retries no longer produce varying expected values or flaky failures.

Written by the indexing model from the issue text.

Description

Summary

message-feedback-snapshots.spec.ts:97 "profile hover uses the channel hover surface" derives its expected value at runtime from a mid-transition sample, so it fails nondeterministically. It has failed on main and on PRs that touch no desktop code.

Evidence: the expectation moves, the received value does not

Desktop Smoke E2E (3), all 3 retries in one job:

retry 0   Expected: "rgba(0, 0, 0, 0.027)"   Received: "rgba(0, 0, 0, 0.04)"
retry 1   Expected: "rgba(0, 0, 0, 0.03)"    Received: "rgba(0, 0, 0, 0.04)"
retry 2   Expected: "rgba(0, 0, 0, 0.016)"   Received: "rgba(0, 0, 0, 0.04)"

A regression moves the received side. Here the received value is stable at the settled 0.04 and the test's own oracle is what varies — three different points on one interpolation curve.

Mechanism (readable from source)

desktop/tests/e2e/message-feedback-snapshots.spec.ts:97-114:

const channel = page.getByTestId("channel-random");
await channel.hover();
const channelHoverColor = await channel.evaluate(
  (element) => getComputedStyle(element).backgroundColor,   // sampled mid-transition
);
await profile.hover();
await expect(profile).toHaveCSS("background-color", channelHoverColor);

await waitForAnimations(page);   // line 110 — AFTER both reads

The channel row's computed background is read immediately after hover(), while the CSS transition toward 0.04 is still interpolating, so it captures an arbitrary intermediate alpha. That value is then asserted against the profile card, which has settled. toHaveCSS retries the profile side, which never changes, so every attempt burns.

The helper that would fix it is already imported and used in this file — it is just on the wrong side of the sample.

Base-branch occurrences (not PR-specific)

run head job result
32074856518 f64899e5d Desktop Smoke E2E (3) same spec, same signature (0.016/0.027/0.016 vs 0.04)
32155633872 17d455a9b (docs-only PR) Desktop Smoke E2E (3) same spec, all 3 retries

Suggested fix

Either await settling before sampling:

await channel.hover();
await waitForAnimations(page);
const channelHoverColor = await channel.evaluate(...);

or assert the literal design token instead of a runtime-sampled value. A test that derives its expected value from live animated state cannot distinguish a real drift from a sample time.

Dominant language
Rust
Stars
33.7k
Forks
4.4k
Avg merge
1d 21h
Merged PRs (30d)
239

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from block/buzz

All issues in block/buzz

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.