Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Optional visual-evidence companion for UI changes: reproducible screenshots without subjective Vitals

Open
  • Is something wrong?
#77 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 9 days

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
playwright, python, storybook
Domain
testing-qa

Research direction

Start by reading the Evidence Observatory, work-scope, and bundle contracts, then trace how consumer artifacts are represented. Draft the ui_evidence.v1 proposal and compare an offline manifest renderer with a consumer narrative; use two PR fixtures for real and flaky diffs. Done means a traceable before/after pair, a missing or bad artifact case, a compatibility note, and an explicit GO/NO-GO decision.

Written by the indexing model from the issue text.

Description

Why this belongs here

Devostasis promises deterministic, model-free repository evidence. Quality of a UI design is a subjective product question and must not become an eighth Vital, a "beauty gauge", a health score, or an LLM claim. However, consumer/report infrastructure may help retain verifiable evidence from UI review. Treat this as a scoped feasibility study for an optional companion, not an approved core-feature expansion.

Upstream OSS / research

  • Playwright, visual screenshot assertions: reproducible browser screenshots and structured failure when a baseline changes.
  • Storybook: named component states, empty/loading/error/focus variations.
  • axe-core: machine-checkable accessibility failures, with the limits of automated auditing stated.
  • React Aria: baseline keyboard/focus behavior to exercise.
  • UI-Bench (paper): pairwise human aesthetic judgements as an explicitly separate evidence class, not a deterministic style score.
  • Penpot MCP: design source provenance if a consuming project already uses it; do not require cloud accounts.

Proposed safe experiment

  • Write a minimal ui_evidence.v1 consumer artifact proposal: repository/commit SHA, exact screenshot/browser/version/viewport/theme, scenario, baseline hash, actual hash, capture result, stale/missing/partial status, comparison mode, and artifact links.
  • Show two deliberately different PR fixtures: genuine visual diff vs a flaky/environment-induced diff. Output must distinguish DIFFERENT, SAME, UNKNOWN, ERROR; never infer 'better-looking'.
  • Sanitize and size-bound text/paths from repositories and external artifacts; respect private repository and screenshot privacy.
  • Compare a purely offline independently reproducible manifest renderer with a consumer-generated narrative; only the evidence hash and deterministic classification may be called verifiable.
  • Confirm architecture compatibility with existing immutable Evidence Observatory/work-scope/bundle contracts, and document rejection if this does not fit cleanly.
  • Do not change the seven Vitals, truth model, public schema, or comparability of historical bundles for this experiment.

Acceptance

One traceable before/after pair, one intentionally missing/bad artifact case, a compatibility note and explicit GO/NO-GO decision. No LLM inside Devostasis core.

Dominant language
Python
Stars
4
Forks
1
Avg merge
5d 12h
Merged PRs (30d)
3

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from drevendev/Devostasis

All issues in drevendev/Devostasis

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.