Wire the voicing lane: register the provider, feed it contracts, calibrate before promoting
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức phù hợp với người mới
- 20/100
- Loại issue
- Tính năng
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức độ hoạt động
- Đình trệ
- Công nghệ
- github, playwright, typescript
- Lĩnh vực
- accessibility, developer-experience, testing
Hướng nghiên cứu
Start with src/deps/build.ts and src/voicing/virtual-sr-provider.ts to trace provider registration, then read src/contracts/index.ts and the existing tests under test/voicing/. The issue sequences config validation, settle-signal integration, Chromium determinism testing, and shared screen-reader calibration before contracts and promotion. Treat #96 item 3 as the canonical calibration reference; done means every listed phase and the stated trust and disclosure requirements are met.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
Relationship to #96
This is the detailed implementation issue behind one bullet of the v0.3.0 tracking issue #96 ("Plus:
wire the voicing lane, which is built, exported, and currently unwired in v0.2.0"). It also shares the
calibration work with item 3 of #96 ("Published Orca calibration"): the calibration harness here (see
Trust question and Phase 4) is the same mechanism, and the two must not fork. Treat #96 item 3 as the
canonical home for the published-calibration attestation and staleness rule; this issue covers the
lane wiring and the engineering the calibration depends on. Held for v0.3.0, no work starts until it
opens.
What the voicing lane is
The lane checks what a screen reader actually says during an interaction, not just whether the page
structure is correct. Axe and the rulepack prove structure: a button has a name, an alert has the
right role, a live region exists. They cannot prove what a blind user experiences during a task: you
delete a cluster, and in the window right after that action the app should announce "Cluster deleted."
That is a claim about a transcript over time, tied to a specific step in a specific interaction. The
voicing lane models that claim and measures it.
An operator declares an InteractionContract for a surface: a task, a list of steps, and a list of
SpeechObligations ("after step N, the announcement must contain these words"). The engine walks the
interaction, records an announcement transcript per stop, and checks the obligations.
What is already built (verified against main at d4afc8f)
The deterministic transcript oracle exists and runs against real Chromium. The gap is input plumbing,
calibration, and authored contracts, not the oracle.
- Frozen contracts in
src/contracts/index.ts:AnnouncementToken(kind, text,fromTree,
source: 'ax-tree' | 'attribute'),TranscriptStop,Step,StepRunner,SpeechObligation,
InteractionContract, andpromotedObligations?: string[]onUsablConfig(empty by default, the
engine never writes it). - The oracle in
src/deps/real.ts: a live-regionMutationObserverover
[aria-live],[role="status"],[role="alert"],[role="log"]witharmAnnouncementCaptureand
drainAnnouncements, plusactiveNodereading the accessibility tree over CDP
Accessibility.getPartialAXTreeandactivePath. Deterministic given the pinned Chromium build,
cross-platform, no application cooperation. - The concrete step runner in
src/providers/keyboard-walk/steps.ts: executes tab, press-escape,
activate, click (throws on unknown kinds), and per step buildsAnnouncementTokens from the active
node and drained live regions with provenance. Already passed into the check runner at
src/deps/build.ts:258. - Structural tier (
src/voicing/structural.ts): deterministic drafts for unreachable focus,
missing name, missing role, missing live announcement, over-long tab path. - Voicing tier (
src/voicing/voicing.ts): word-levelvoicing/missing-announcement, preview and
unverifiedby default, deterministicfailonly for promoted obligation classes. Matcher in
src/voicing/normalize.ts. - The provider (
src/voicing/virtual-sr-provider.ts):makeVirtualSrProvider(contracts, stepRunner), runs both tiers per matching surface, readspromotedObligations, never self-promotes. - Tests:
test/voicing/{matching,normalize,promotion-invariants,structural,virtual-sr-provider,voicing}.test.ts.
What is not wired (the precise gaps)
- The provider is never registered.
makeVirtualSrProvideris exported fromsrc/index.tsbut
not in the providers array atsrc/deps/build.ts:222, so it never runs duringusabl check. - No config input for contracts.
UsablConfighas no field, loader, or validation for
InteractionContract[]. The provider takes contracts but nothing supplies them. Main missing
plumbing. - No contracts are authored for any surface.
- No calibration (shared with #96 item 3). The oracle is deterministic but unproven against a
real screen reader, so no obligation class can be promoted honestly yet. - The capture window is not bound to a settle signal.
drainAnnouncementsruns right after each
step, so a late live-region update can be missed and timing can vary run to run. Ties to the #146
readiness work and the #186 partial-render residual.
The trust question: deterministic versus trustworthy
The oracle is deterministic. It is not yet proven trustworthy. Trustworthy can only mean "agrees with
a named reference screen reader within a measured bound," never "universally correct," because screen
readers genuinely disagree across versions, settings, and platforms.
You earn the bound with a calibration harness, run offline, never in the gate path: drive the same
interaction fixtures with a real screen reader (Orca on Linux per #96 item 3, and
@guidepup/guidepup for NVDA and VoiceOver), capture the spoken phrases, compare to the
reconstruction at the token level (not exact strings, which is what obligationSatisfied already
does), and produce a measured agreement number per obligation class. That number is the evidence that
lets a class be promoted. Where agreement is too low, the class stays preview and the engine abstains.
Plan
Sequenced. Each phase complete to the bar before the next. Nothing gates on a signal it cannot back.
- Phase 0, spike the trust question. A handful of fixtures, reconstruction versus a real screen
reader, measured token agreement. This decides whether the reconstruction is faithful enough to ever
promote. If no, the lane ships as an advisory preview signal only and later phases scope down. - Phase 1, contract input path. Add interaction contracts to config, validated and trusted-ref
overlaid the same way requirements are, so a pull request cannot rewrite its own obligations and
pass itself. Validate steps non-empty,afterStepin range,requiredTokensnon-empty,surfaceId
matches a declared surface. Additive, contracts are already frozen. - Phase 2, register the provider. Add
makeVirtualSrProvider(contracts, stepRunner)to the
providers array insrc/deps/build.ts, present only when contracts are configured. It declares
capabilities: ['live'], so under static-only mode it records not-covered gaps, not a crash.
Confirm drafts reach the gate as preview (non-gating) by default. - Phase 3, determinism hardening. Bind the announcement capture window to the #146 settle signal so
drainAnnouncementsis stable and late updates are not missed. Add a real-Chromium integration test
that runs a known interaction twice and asserts an identical transcript. - Phase 4, calibration harness (shared with #96 item 3). Turn the Phase 0 spike into a repeatable
tool: fixtures, Orca and guidepup capture, token-level comparison, a measured agreement report per
obligation class, stored as the justification for any promotion. - Phase 5, author contracts and promote. Write
InteractionContracts for the real surfaces.
Promote only the classes the calibration report backs. Disclose the reference screen reader and the
agreement bound on every promoted finding, asfixtext is framed as untrusted today. - Phase 6, surfaces and docs. Render voicing findings on the stop hook, the dev overlay, and CI and
pull request comments, advisory where preview and deterministic where promoted. Same disclosure
discipline as the rest of the engine.
Risks and open decisions
- Screen readers disagree. Trust is a measured bound against a named reference, never universal.
Disclose it on the finding. - Live-region coalescing. The
MutationObservercaptures every mutation; a real screen reader
drops some under load, so the reconstruction can over-report announcements a user would miss. An
optimism risk on "was it announced," which calibration measures. - Browse mode versus focus mode. Interaction contracts are focus and action based, which maps to
focus mode. Browse-mode reading is out of scope and should be stated. - Authoring cost (decision). Contracts are operator content. Accept hand-authored contracts, or
auto-derive candidate contracts from discovery for the operator to confirm. - Where contracts live (decision). Inline in
usabl.config.json, a referenced file, or folded
into the requirements bundle. - guidepup in CI. Drives real screen readers on macOS and Windows runners and can be flaky.
Acceptable because it is the offline calibration reference, never the runtime oracle.
Definition of done
- The provider is registered and runs on live scans when contracts are configured, abstains (preview)
by default, and records not-covered under static-only mode. - Config carries interaction contracts with validation and trusted-ref overlay.
- The transcript is proven deterministic by a real-Chromium test that runs twice with identical output.
- The calibration harness produces a measured agreement per obligation class, and only backed classes
are promoted, with the reference screen reader and the bound disclosed (in step with #96 item 3). - Voicing findings render on all surfaces with the correct trust framing.
- No false green: an unmeasured or uncalibrated obligation never gates.
- Ngôn ngữ chính
- TypeScript
- Star
- 0
- Fork
- 0
- Merge trung bình
- 2 giờ 48 phút
- Pull request đã merge (30 ngày)
- 8
Chuẩn bị môi trường
- Không có Dockerfile hay tệp Docker Compose
- Có mẫu pull request
- Không có hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của usabl-dev/usabl
-
area: engine bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 86/100
-
Stop hook allows a stop on an unknown verdict, and the self-check prints IDLE for a null verdictĐang mởarea: assistants bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 75/100
-
area: engine bug security
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
-
Overlay: browser context failures are counted per plugin, not per browser, and close is not sharedĐang mởarea: overlay bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 64/100
-
area: engine enhancement performance
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
Tất cả issue của usabl-dev/usabl
Issue tương tự
-
Độ khó 1/5 1-3 giờ Mức phù hợp với người mới 82/100
source-cooperative/docs.source.coop#41 · 1 bình luận ·
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 76/100
Nimblesite/SharpLsp#468 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 85/100
solidjs-community/solid-primitives#1082 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
enhancement
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 64/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
Maintainer thường phản hồi trong vòng 1 ngày