intermittent test failures: web-forms e2e: firefox visuals
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- Half a day
- Newbie friendliness
- 54/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- playwright, typescript
- Domain
- testing
Research direction
Start with the failing visual test at e2e/test-cases/visual/all-question-types.test.ts:68 and the linked GitHub Actions run to inspect the Firefox screenshot mismatch for select-map-zoom-out.png. Check the screenshot helper at e2e/screenshot.ts:102 and the map control at e2e/page-objects/controls/MapControl.ts:216. Done means identifying and fixing the intermittent failures so the affected Firefox visual tests pass reliably.
Written by the indexing model from the issue text.
Description
https://github.com/getodk/central-frontend/actions/runs/43368646506/job/131701057077?pr=1990
Running 21 tests using 1 worker
··××F××F·········×±×±······
1) [visual-firefox] › e2e/test-cases/visual/all-question-types.test.ts:68:5 › All Question Types (Visual) › Select one from map › zooms in and out, pans the map and zooms to fit all features
Error: expect(page).toHaveScreenshot(expected) failed
30275 pixels (ratio 0.08 of all image pixels) are different.
Snapshot: select-map-zoom-out.png
Call log:
- Expect "toHaveScreenshot(select-map-zoom-out.png)" with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 30275 pixels (ratio 0.08 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 30275 pixels (ratio 0.08 of all image pixels) are different.
at screenshot.ts:102
100 | const box = await getBox(locator, screenshotName);
101 |
> 102 | await expect(page).toHaveScreenshot(screenshotName, {
| ^
103 | fullPage: isFullPage,
104 | clip: {
105 | x: Math.round(box.x + scroll.x),
at expectScreenshot (/__w/central-frontend/central-frontend/packages/web-forms/e2e/screenshot.ts:102:22)
at MapControl.expectMapScreenshot (/__w/central-frontend/central-frontend/packages/web-forms/e2e/page-objects/controls/MapControl.ts:216:5)
at /__w/central-frontend/central-frontend/packages/web-forms/e2e/test-cases/visual/all-question-types.test.ts:70:7
attachment getodk/central#1: select-map-zoom-out (image/png) ─────────────────────────────────────────────────
Expected: e2e/test-cases/visual/all-question-types.test.ts-snapshots/select-map-zoom-out-firefox.png
Received: test-results/test-cases-visual-all-ques-481b5-d-zooms-to-fit-all-features-visual-firefox/select-map-zoom-out-actual.png
Diff: test-results/test-cases-visual-all-ques-481b5-d-zooms-to-fit-all-features-visual-firefox/select-map-zoom-out-diff.png
Error Context: test-results/test-cases-visual-image-la-f28e5-cked-columns-and-no-buttons-visual-firefox/error-context.md
2 failed
[visual-firefox] › e2e/test-cases/visual/all-question-types.test.ts:68:5 › All Question Types (Visual) › Select one from map › zooms in and out, pans the map and zooms to fit all features
[visual-firefox] › e2e/test-cases/visual/all-question-types.test.ts:82:5 › All Question Types (Visual) › Select one from map › opens details of saved feature and remove saved feature
2 flaky
[visual-firefox] › e2e/test-cases/visual/image-labels.test.ts:53:5 › Image Selects (Visual) › Select with images › select one with fixed column count
[visual-firefox] › e2e/test-cases/visual/image-labels.test.ts:66:5 › Image Selects (Visual) › Select with images › select one with packed columns and no buttons
17 passed (2.8m)
Error: Process completed with exit code 1.
- Dominant language
- TypeScript
- Stars
- 38
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from getodk/web-forms
-
iOS
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Offline send queueOpenIn Enketo
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Launch form offlineOpenneeds discussion
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Progress barOpenIn Enketo needs ui/ux design
Difficulty 2/5 1-3 hours Newbie friendliness 58/100
All issues in getodk/web-forms
Similar issues
-
area:plugins kind:bug priority:p2 ready
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
Maintainers usually reply within 1 day
-
Escaped `*` followed by `*` (`\**`) never matchesPossibly taken @bardrop claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 71/100
editorconfig/editorconfig-core-js#172 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
vega/vega-webgpu#40 ·
-
type: bug 🐛
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
puckeditor/puck#1929 ·
Maintainers usually reply within 1 day
-
compression() compresses every response at Brotli quality 11Possibly taken @staylor claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day