Setup PR cannot bootstrap its first baseline
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- github-actions, javascript, playwright
Research direction
Start with the documented setup flow, the report mode, and the Frameshift action’s exact-base-SHA baseline lookup described in the issue. Reproduce the first setup PR and the case where the default-branch artifact is still pending, using the linked integration evidence and the 61 Playwright tests as references. Done means setup CI does not fail when no baseline can yet exist, while later comparisons still wait for and use an available baseline.
Written by the indexing model from the issue text.
Description
The documented setup flow has a chicken-and-egg failure. A repository’s first Frameshift setup PR runs in report mode, waits about three minutes for an artifact tied to the exact base commit, then fails because that commit could not have produced a Frameshift baseline before the workflow existed.
Observed with dcramer/frameshift/ci@68a8b5e8bbd439088ef9a044e693c5de9efe7ecd:
- All 61 Playwright tests passed and produced 26 PNG files.
Find the before screenshotswaited for 180 seconds.- It failed with
No saved screenshots named frameshift-baseline-v1-<base sha> were found. - The setup guide says to merge the workflow and let it run on the default branch before the first PR. That cannot make the setup PR itself pass when CI is required.
- The exact-base-SHA lookup also creates a race for PRs opened before the corresponding default-branch workflow finishes. The action waits, but then fails if the baseline is still unavailable.
- The failure happens late in the job and looks like a screenshot or E2E failure even though capture succeeded; the required CI check then fails too.
Current downstream workaround: query Actions for a live baseline artifact before invoking Frameshift, skip the comparison only when no baseline exists, and let the first default-branch run create it.
Related integration PR and failure evidence: https://github.com/getsentry/junior/pull/1753 and https://github.com/getsentry/junior/actions/runs/33536953468/job/99953603443
Requested by David Cramer.
--
- Dominant language
- JavaScript
- Stars
- 4
- Forks
- 0
- Avg merge
- 16m
- Merged PRs (30d)
- 1
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Similar issues
-
bug user-priority/P2
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
bug confirmed perf
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
videojs/video.js#9400 · 1 comment ·
Maintainers usually reply within 1 day
-
agent/scanner bug hive/hosted-available-lke648397-260827-5n31
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
rescript-lang/rescript#8765 ·
Maintainers usually reply within 1 day
-
feedback simulation workshop
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
githubnext/gh-aw-workshop#4417 ·
Maintainers usually reply within 1 day