The agentic layer's repeatable steps moved from the model into scripts

Aperta
  • Qualcosa non va?
#112 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
10/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
angular, playwright, python
Ambito
testing-qa, tooling

Direzione di ricerca

This is an epic with several sub-tasks, so start with the first one: re-audit the four skills and two scripts under agentic/.claude/skills/ against the 2026-09 plan. Then read living-doc-shared/lib/ and the compute_gaps.py and trace_impact.py scripts, plus agentic/CONTRIBUTING.md and agentic/docs/testing/eval-protocol.md. Done means the audit is recorded and a single sub-task is ready to start, not the whole epic.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

type:epic
Background

Several skills in agentic/ ask the model to do work a script could do the same way every time: comparing browser
snapshots, resolving routes, linting step definitions, matching steps to definitions. Every step runs on whatever model
the session uses, so a cheap scan costs as much as a judgement, and answers vary between runs where a script would give
one result. A plan written in 2026-09, when the layer still lived in agentic-toolkit, audited four skills and
specified their scripts. Its paths are from before the move to agentic/.claude/skills/, and some skills have changed
since.

Goals
  • Every script prints one JSON envelope that says whether its result is final, ambiguous (with narrowed candidates,
    not a raw dump) or an error, so a skill knows when the model is needed.
  • The four skills' mechanical steps run as scripts with tests.
  • Each step names the model it needs: a small model scans, a large model judges.
  • Token cost and answer quality are measured before and after, with a go/no-go before the pattern spreads.
Subtasks

In landing order. The script specs below come from the 2026-09 audit; the first child re-checks them.

  • Re-audit the four skills and the two scripts to retrofit, against agentic/.claude/skills/ today.
  • Escalation contract: the envelope's schema and reference page, plus a helper module in living-doc-shared/lib/.
    living-doc-gap-finder's compute_gaps.py and living-doc-impact-analysis's trace_impact.py adopt it
    first, and agentic/CONTRIBUTING.md says every new script speaks it.
  • living-doc-pageobject-scan: summarize_snapshot.py. It condenses one MCP Playwright accessibility snapshot
    into a per-route element table and a new-versus-known diff, so the crawl stops re-reading raw snapshots. It
    appends each iteration's counts to a scratch log and flags a coverage plateau (no new elements or routes over N
    iterations); the plateau check lives here, not in manifest_diff.py.
  • data-cy-instrument: two scripts. resolve_gap_targets.py (Angular only) maps a manifest coverage gap and its
    route to the component file through the routing modules ({ path, component } and lazy loadComponent
    entries, route parameters, feature-flag variants). validate_test_id_name.py is framework-agnostic and checks a
    proposed test id: kebab-case, then the naming rules.
  • gherkin-step: lint_steps.py, a static lint of step definitions.
  • gherkin-living-doc-sync: the step matching in bdd-maintain's find_unused_steps.py (cucumber_to_regex,
    StepDefinition, collect_step_definitions with a file-glob parameter) moves into
    living-doc-shared/lib/step_matcher.py. A new detect_step_drift.py finds .feature step lines with no
    matching definition and reports, for each, the most similar definition (a difflib ratio of at least 0.5) and
    the PageObject method it calls.
  • Model tiering: the agent's mode dispatch names the model per step. A two-phase pattern (cheap scan, expensive
    judgement) goes into pageobject-scan's create and re-scan, bdd-maintain's dead-code audit and gap-finder's
    audit, and a guide states the pattern.
  • Measurement: define benchmark sessions, capture a baseline before tiering and a capture after it, then a
    comparison with a go/no-go, and a retro with the follow-ups.
Related / References
  • agentic/docs/testing/eval-protocol.md, agentic/docs/testing/baseline.md (how a change is measured).
  • Related issues: living-doc-gap-finder's compute_gaps.py defects (fix before the retrofit); the eval tooling.
  • Not in this epic: PageObject generation and healing, Playwright crawling and step-definition authoring. They stay
    out, and none of the agentic layer is ever a pipeline dependency.
Lingua principale
Python
Stelle
0
Fork
0
Merge medio
6h 14m
PR unite (30g)
27

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di AbsaOSS/living-doc

Tutte le issue di AbsaOSS/living-doc

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.