Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Add Playwright end-to-end tests for the extension's UI, following Positron's e2e pattern

Aperta
  • Qualcosa non va?
#1,177 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
48/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
playwright, typescript
Ambito
ci-cd, developer-experience, testing-qa

Direzione di ricerca

Start by comparing Positron's test/e2e layout with the existing yarn test-positron and @vscode/test-electron setup; use the proposed apps/vscode/test/e2e or apps/vscode-e2e and playwright.config.ts as entry points. Run npx playwright test --project e2e-electron once the driver exists, and consider the milestone complete when the project, CI HTML report, visual-editor specs, and test-author README or CLAUDE.md are in place.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

Our end-to-end tests (yarn test-vscode, yarn test-positron) run inside the extension host and drive VS Code through its API. They can open documents, run commands, and query symbols, completions and diagnostics. They can't see anything rendered in a webview, and that's where a lot of the extension lives:

  • The visual editor (ProseMirror, and CodeMirror in code cells). The roundtrip tests in quartoDoc.test.ts check the markdown that comes back, using quarto.test_isVisualEditorSynced (#1167), but nothing can check the cell toolbar, syntax highlighting, autocomplete, search, spelling underlines, dialogs, or the outline. The two visual-editor tests in quartoEditor.test.ts are test.skip.
  • The notebook renderer and the help panel.
  • The packaged VSIX. CI tests the dev build, not the artifact we publish.

So every dependency PR that touches these areas ends with a "manual smoke test" checklist. CodeMirror 6.43 in #1176 is a recent example, and ProseMirror and React upgrades are next. Pure rendering logic is being moved into snapshot tests (#1174); this issue covers the interactive UI.

Proposal: follow the Positron e2e pattern

posit-dev/positron's test/e2e uses Playwright Test to drive the real Positron app, as Electron or in a browser. We'll set up the same structure here, driving VS Code (and Positron) with our extension installed. What that pattern consists of:

  • Layout. Everything lives under a single test directory:
    • infra/: the app driver (launch, Electron/browser/external-server modes, logging, teardown).
    • pages/: one page-object class per area, which owns the selectors and actions for that area.
    • tests/<area>/: specs, one directory per feature.
  • Driver. Positron's infra/ descends from VS Code's own smoke-test automation driver (playwrightDriver.ts, playwrightElectron.ts, code.ts); those files carry Microsoft's MIT header.
  • Fixtures. A shared tests/_test.setup.ts extends Playwright's test with fixtures such as app, page, openFile, settings/settingsFile, hotKeys, runCommand, logger, and automatic screenshot, trace and log attachment. Each area can have its own _test.setup.ts that extends it with its own settings; tests/quarto/_test.setup.ts enables quarto.inlineOutput.enabled, for example.
    • Each spec file declares test.use({ suiteId: __filename }), so it gets a fresh app instance.
    • Workspaces are provisioned from an in-repo test-files/ directory.
    • There's a commented example spec (tests/example.test.ts) and a CLAUDE.md for test authors.
  • Playwright projects for each target: e2e-electron (the default), e2e-chromium/e2e-server (browser), and e2e-windows/e2e-macOS-ci, filtered by tag.
  • Tags. Tests carry tags such as @:critical, @:quarto and @:web. @:critical always runs on PRs. Other tags run when they appear in the PR description, or when changed paths map to them (.github/workflows/test-tag-paths-map.json).
  • CI. Retries once in CI and not locally; 3 workers; 2-minute test and 15 s expect timeouts. Traces, screenshots and logs are captured on failure and published as an HTML report artifact.
  • Tests are run and debugged locally from the command line (npx playwright test --project e2e-electron), UI mode, or the Playwright Test extension.
Adapting it here
  • Put it in a new workspace, e.g. apps/vscode/test/e2e or a separate apps/vscode-e2e, with its own playwright.config.ts.
  • Launch VS Code (downloaded with @vscode/test-electron, which we already use) with the extension, preferably the packaged VSIX so we test what ships. Add a Positron project that reuses the download logic from yarn test-positron.
  • Start with a small set of page objects: visualEditor (toggle mode, the cell toolbar, typing in a code cell, completion popups, spelling marks), outline, helpPanel, and notebookRenderer. For locators in our own webviews, add data-testid attributes rather than rely on generated class names, as Positron does ([data-testid="quarto-kernel-status"]).
  • Start with a @:critical set that runs on every PR, and path-mapped tags later if the suite grows.
  • Scope out a browser-only project for the editor bundle (apps/vscode-editor, with the host messages stubbed) as a possible faster lane. It would also cover panmirror for RStudio.

Things to settle

  • Licensing. Positron's e2e tests, page objects and fixtures are under the Elastic License 2.0; this repo is MIT. We can follow the structure and conventions freely, but copying Positron-authored code needs a decision from Posit. VS Code's MIT-licensed smoke-test driver files can be used either way.
  • Overlap with Positron's own tests. Positron already has tests/quarto/ specs that exercise this extension inside Positron: inline output, the outline, the Python and R cells. We should coordinate so we don't duplicate them. Ideally a Positron-specific regression found here becomes a test there, and the reverse.
  • CI cost. Decide which OS runs on every PR, and whether R, Python and the Quarto CLI need installing in CI for the first tests. The first visual-editor tests shouldn't need any of them.

First milestone

  • The driver, one e2e-electron project, and CI wired up with an HTML report artifact.
  • Visual editor specs that cover what the tranche 4 PRs (#1174, #1176) left to manual testing:
    • open a .qmd in visual mode;
    • edit a code cell, check highlighting, and use the cell toolbar;
    • trigger autocomplete;
    • check spelling marks;
    • switch back to source mode, and check the text roundtrips.
  • The CLAUDE.md / README for test authors, including how to run a single spec and open the trace.

🤖 Generated with Claude Code

Lingua principale
TypeScript
Stelle
655
Fork
64
Merge medio
8h 45m
PR unite (30g)
36

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di quarto-dev/quarto

Tutte le issue di quarto-dev/quarto

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.