stardust:replica gates against page-mode=static fallback, not the real rendering, on virtual-scroll pages
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- javascript
- Domain
- devtools
Research direction
Start by reading source-fidelity-gate.md, especially hardening rule 13, then trace how capture mode and gate results are recorded in progress.json and shown in progress.html. The acceptance criteria require enforcing fallback-mode checks, comparing app viewports, and surfacing reference details, so identify the relevant capture and reporting entry points before estimating the change. Done means a fallback cannot silently serve as the live side, and the final summary and status page identify each gate's reference URL and mode.
Written by the indexing model from the issue text.
Description
Summary
stardust:replica migrated an Adobe Express webpage by capturing its screen-reader-friendly fallback (?page-mode=static) and using that as both the design source and the "live side" of every fidelity gate. The gates passed (1440: 2.34%, 360: 5.21%, ai-readability 100, "Ready 100%"), but the migrated page does not resemble the page a visitor sees. The run never told the user the reference was a different rendering.
Run: stardust:replica --hands-off, single page (--single --dynamics), plugin 0.27.0+e68296b7aee4, model claude-fable-5-1, 6 sessions, ~$60.
- Source: https://new.express.adobe.com/webpage/DYft29Wzo0QLK
- Result: https://main--b7aaf274--aemcoder.aem.page/
What happened
- Capture. The Express page is a virtual-scroll app: every
.sectionisposition:absolute, translated by JS andhiddenoff-screen;document.scrollHeight= 900 andwindow.scrollTois a no-op. - The agent recognised this (journal, Phase 1) and cited
source-fidelity-gate.md§ Hardening rule 13. Rule 13 says to record the page as gate-blocked for the pixel probe and rely on content-diff/visual-diff. - It did not follow rule 13. It captured
?page-mode=staticinstead (9327px, stacked layout) and promoted it to the canonical page record. The real app-mode capture was kept only aspages/<slug>-app.{json,html}andscreenshots/<slug>-app.png, and was never used as a reference. - Every downstream step used the fallback.
direction.md,progress.json#liveSide, the prototype CSS ("noscript @1084–@2181: flex column, photo 50vh above or below the text"), and the published-origin gate, whose origin side was seeded from the static-modelive.png. - Handoff. The final summaries say "same design", "Ready 100%", "no gaps". They do not say the reference was the accessibility rendering.
Observed difference (screenshots at 1440x900)
| Real page | Static fallback / migrated EDS page | |
|---|---|---|
| Hero | full viewport (900px) beach photo, wide rounded display title | 720px (80% of the article), narrower title |
| Text + photo sections | 50/50 split: text left, food photo fills the right half | single narrow text column, photo as a thin ~40-70px strip |
| Photo bands | full-bleed | cropped strips |
| Display font | Poplar Std (wide, heavy) | Anton (condensed) |
Root causes
- Rule 13's gate-blocked outcome is bypassed. Nothing stops the agent substituting an alternative render mode (
page-mode=static,noscript, reader view). Static mode is a different layout, not "the same publication with the same theme". The agent's own phrasing ("renders the same publication, theme and fonts") was never verified visually against app mode. - No gate row compares against the real app viewport. The agent had a 1440x900 app-mode screenshot from the crawl and never diffed against it. Content-diff and visual-diff both ran against the fallback.
- The Anton substitution is described as "metric-matched" (R-02) but is not. Poplar Std is much wider than Anton. The gate classifies heading bands as a "justified flag zone", and the 2560 cap-probe failure (395px vs 417px) is waved through as register-justified. The metric claim was never measured.
- Reference choice is invisible to the user.
progress.html, the status page and the final summary report pass/fail only; the reference URL and mode are not surfaced. - Cost (secondary). The runner answered
stopat most phase boundaries, citing the context budget at least once. Each fresh session re-read the replica card and journal sections, which contributed to ~$60 and ~54M cache-read tokens for a single page.
Proposed changes
- Make rule 13 enforceable. When capture detects an inner-scroller page and falls back to any alternative render mode, require: (a) a visual check of that mode against the app-mode viewport screenshot, and (b) if layout differs (hero height, split vs stacked, section order), mark the breakpoint gate-blocked and surface it, rather than gating against the alternate mode.
- Add an app-viewport gate row for virtual-scroll pages: compare the first viewport (and a scripted sequence of scroll positions where the app supports it) at 1440x900 and 360x800 between source and build. Capturing the inner scroller is already listed as future work in rule 13.
- Record the reference mode (
liveSide, capture mode, whether it is the user-facing rendering) inprogress.jsonand the status page, and print it in the final summary. A gate pass against a non-user-facing mode must read as "passed against fallback rendering", not "Ready 100%". - Measure font substitutes. Compute glyph advance widths (or a rendered-width ratio over the actual heading strings) for the licensed face vs the substitute, and flag when the ratio exceeds a threshold, instead of labelling the substitute "metric-matched". Candidate fix when the face is not rehostable: a wider OFL substitute, or
size-adjust/letter-spacingtuning. - Review runner stop policy.
stopat nearly every boundary on a one-page job multiplies orientation cost. Consider a larger budget or batching adjacent small phases for--singleruns.
Acceptance criteria
- On this page, the run either (a) produces a build whose first viewport matches the app-mode screenshot within the gate threshold, or (b) ends with the pixel gate marked blocked and the reason printed in the final summary.
- A fallback render mode can never be recorded as the gate's live side without a recorded visual-equivalence check.
- Final summary and
progress.htmlstate which URL/mode each gate compared against.
Evidence
Full LLM transcript of the run is available from Labs (OF1 Labs experiment b7aaf274-67bd-4ff5-b6e8-76a7beab3c93, GET /api/experiments/:id/claude-log); key entries are cited above by session step. I can attach the real vs migrated screenshots on request.
- Dominant language
- JavaScript
- Stars
- 324
- Forks
- 99
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 121
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from adobe/skills
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
Similar issues
-
nip27: long hashtag swallows the rest of the content, lone # gives an empty hashtagPossibly taken @bardrop claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
nbd-wtf/nostr-tools#578 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
h5bp/html5-boilerplate#3512 ·
-
documentation
Difficulty 1/5 1-3 hours Newbie friendliness 85/100
githubnext/gh-aw-workshop#5752 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 70/100
Maintainers usually reply within 1 day