Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Add Playwright end-to-end tests for the extension's UI, following Positron's e2e pattern

Abierto
  • ¿Algo no está bien?
#1,177 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
48/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
playwright, typescript
Área
ci-cd, developer-experience, testing-qa

Línea de trabajo

Start by comparing Positron's test/e2e layout with the existing yarn test-positron and @vscode/test-electron setup; use the proposed apps/vscode/test/e2e or apps/vscode-e2e and playwright.config.ts as entry points. Run npx playwright test --project e2e-electron once the driver exists, and consider the milestone complete when the project, CI HTML report, visual-editor specs, and test-author README or CLAUDE.md are in place.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Problem

Our end-to-end tests (yarn test-vscode, yarn test-positron) run inside the extension host and drive VS Code through its API. They can open documents, run commands, and query symbols, completions and diagnostics. They can't see anything rendered in a webview, and that's where a lot of the extension lives:

  • The visual editor (ProseMirror, and CodeMirror in code cells). The roundtrip tests in quartoDoc.test.ts check the markdown that comes back, using quarto.test_isVisualEditorSynced (#1167), but nothing can check the cell toolbar, syntax highlighting, autocomplete, search, spelling underlines, dialogs, or the outline. The two visual-editor tests in quartoEditor.test.ts are test.skip.
  • The notebook renderer and the help panel.
  • The packaged VSIX. CI tests the dev build, not the artifact we publish.

So every dependency PR that touches these areas ends with a "manual smoke test" checklist. CodeMirror 6.43 in #1176 is a recent example, and ProseMirror and React upgrades are next. Pure rendering logic is being moved into snapshot tests (#1174); this issue covers the interactive UI.

Proposal: follow the Positron e2e pattern

posit-dev/positron's test/e2e uses Playwright Test to drive the real Positron app, as Electron or in a browser. We'll set up the same structure here, driving VS Code (and Positron) with our extension installed. What that pattern consists of:

  • Layout. Everything lives under a single test directory:
    • infra/: the app driver (launch, Electron/browser/external-server modes, logging, teardown).
    • pages/: one page-object class per area, which owns the selectors and actions for that area.
    • tests/<area>/: specs, one directory per feature.
  • Driver. Positron's infra/ descends from VS Code's own smoke-test automation driver (playwrightDriver.ts, playwrightElectron.ts, code.ts); those files carry Microsoft's MIT header.
  • Fixtures. A shared tests/_test.setup.ts extends Playwright's test with fixtures such as app, page, openFile, settings/settingsFile, hotKeys, runCommand, logger, and automatic screenshot, trace and log attachment. Each area can have its own _test.setup.ts that extends it with its own settings; tests/quarto/_test.setup.ts enables quarto.inlineOutput.enabled, for example.
    • Each spec file declares test.use({ suiteId: __filename }), so it gets a fresh app instance.
    • Workspaces are provisioned from an in-repo test-files/ directory.
    • There's a commented example spec (tests/example.test.ts) and a CLAUDE.md for test authors.
  • Playwright projects for each target: e2e-electron (the default), e2e-chromium/e2e-server (browser), and e2e-windows/e2e-macOS-ci, filtered by tag.
  • Tags. Tests carry tags such as @:critical, @:quarto and @:web. @:critical always runs on PRs. Other tags run when they appear in the PR description, or when changed paths map to them (.github/workflows/test-tag-paths-map.json).
  • CI. Retries once in CI and not locally; 3 workers; 2-minute test and 15 s expect timeouts. Traces, screenshots and logs are captured on failure and published as an HTML report artifact.
  • Tests are run and debugged locally from the command line (npx playwright test --project e2e-electron), UI mode, or the Playwright Test extension.
Adapting it here
  • Put it in a new workspace, e.g. apps/vscode/test/e2e or a separate apps/vscode-e2e, with its own playwright.config.ts.
  • Launch VS Code (downloaded with @vscode/test-electron, which we already use) with the extension, preferably the packaged VSIX so we test what ships. Add a Positron project that reuses the download logic from yarn test-positron.
  • Start with a small set of page objects: visualEditor (toggle mode, the cell toolbar, typing in a code cell, completion popups, spelling marks), outline, helpPanel, and notebookRenderer. For locators in our own webviews, add data-testid attributes rather than rely on generated class names, as Positron does ([data-testid="quarto-kernel-status"]).
  • Start with a @:critical set that runs on every PR, and path-mapped tags later if the suite grows.
  • Scope out a browser-only project for the editor bundle (apps/vscode-editor, with the host messages stubbed) as a possible faster lane. It would also cover panmirror for RStudio.

Things to settle

  • Licensing. Positron's e2e tests, page objects and fixtures are under the Elastic License 2.0; this repo is MIT. We can follow the structure and conventions freely, but copying Positron-authored code needs a decision from Posit. VS Code's MIT-licensed smoke-test driver files can be used either way.
  • Overlap with Positron's own tests. Positron already has tests/quarto/ specs that exercise this extension inside Positron: inline output, the outline, the Python and R cells. We should coordinate so we don't duplicate them. Ideally a Positron-specific regression found here becomes a test there, and the reverse.
  • CI cost. Decide which OS runs on every PR, and whether R, Python and the Quarto CLI need installing in CI for the first tests. The first visual-editor tests shouldn't need any of them.

First milestone

  • The driver, one e2e-electron project, and CI wired up with an HTML report artifact.
  • Visual editor specs that cover what the tranche 4 PRs (#1174, #1176) left to manual testing:
    • open a .qmd in visual mode;
    • edit a code cell, check highlighting, and use the cell toolbar;
    • trigger autocomplete;
    • check spelling marks;
    • switch back to source mode, and check the text roundtrips.
  • The CLAUDE.md / README for test authors, including how to run a single spec and open the trace.

🤖 Generated with Claude Code

Lenguaje dominante
TypeScript
Estrellas
652
Forks
64
Merge medio
8 h 45 min
PR fusionados (30 d)
36

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de quarto-dev/quarto

Todos los issues de quarto-dev/quarto

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.