Run the validator in the browser from GitHub Pages, pointed at a public repository (Pyodide)

Aberta
  • Algo errado?
#127 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
5/5
Tempo estimado
Mais de uma semana
Facilidade para iniciantes
35/100
Tipo de issue
Funcionalidade
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
javascript, python
Domínio
devtools, web-dev

Direção de pesquisa

Start with the Pyodide spike described in the issue and the validator CLI entry point, then build the local-folder flow in web/index.html and web/worker.js. The proposal gives a staged path through GitHub API and raw file fetching, rendering, permalinks, and CI deployment to GitHub Pages. Done means the browser can validate a local folder and public repository and display the validator's JSON output.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Proposal

A static page on GitHub Pages that runs the validator in the visitor's browser (no server), pointed at any public GitHub repository (?repo=owner/name@ref) or at a local folder. Feasibility was investigated on 2026-11-08 against rearchitecture (#126, head 9d27689); findings below. The validator itself needs no change.

1. The validator runs unmodified under Pyodide

Pyodide is CPython compiled to WebAssembly. [email protected] ships Python 3.14.2 (the package version now tracks Python's), so requires-python >= 3.14 and the 3.14-only syntax in the code base (except ValueError, re.error:) are fine, and it ships a wasm wheel of PyYAML 6.0.3, the one dependency.

Spike: build the wheel (pip wheel . → project_validator-0.1.0-py3-none-any.whl, 340 KB, pure Python), load Pyodide in node, micropip.install the wheel, mount a checkout, run python -m project_validator <dir> --format json with stdout captured.

Repository Files Native Pyodide Output
cons_expr 73 0.7 s 2.4 s byte-identical JSON
tokio 874 2.7 s 8.5 s byte-identical JSON
prometheus 1742 8.8 s 24.4 s byte-identical JSON

About 3× native. One-time runtime load is ~13 MB (pyodide.asm.wasm 9.2 MB, stdlib 2.5 MB, PyYAML, the wheel), browser-cached afterwards; Pyodide is on the jsDelivr CDN so Pages hosts only the page and the wheel.

  • git is absent: shutil.which("git") is None, so Project.tracked_files() falls back to every file. For a snapshot of a repository that is the correct answer (a snapshot has no untracked files), the same as validating a tarball.
  • --build (cmake/ctest through subprocess) cannot work in a browser and would be hidden. Everything else is static reading.
Spike script (node) 
import { loadPyodide } from "pyodide";
const [,, wheel, target, ...args] = process.argv;
const pyodide = await loadPyodide();
await pyodide.loadPackage(["pyyaml", "micropip"]);
pyodide.FS.mkdirTree("/wheels");
pyodide.mountNodeFS("/wheels", wheel.slice(0, wheel.lastIndexOf("/")));
await pyodide.pyimport("micropip").install("emfs:/wheels/" + wheel.slice(wheel.lastIndexOf("/") + 1));
pyodide.FS.mkdirTree("/proj");
pyodide.mountNodeFS("/proj", target);
pyodide.runPython(`
import sys, runpy, io
sys.argv = ["project-validator", "/proj", *${JSON.stringify(args)}]
out = io.StringIO(); sys.stdout = out
try:
    runpy.run_module("project_validator", run_name="__main__")
except SystemExit as e:
    code = e.code
sys.stdout = sys.__stdout__
open("/wheels/out.json", "w").write(out.getvalue())
`);

2. Getting a repository into the page — verified in a real browser

Tested from a page served on http://127.0.0.1:8765 (a different origin from GitHub, as a Pages site is) in headless Firefox, fetching cons_expr, tokio and pydantic. Browsers enforce CORS; curl does not, so this is the test that counts.

Source From the page Rate limit Note
api.github.com/repos/o/r/tarball, codeload.github.com/…/tar.gz, github.com/…/archive/….tar.gz fails (NetworkError; codeload's CORS allows only render.githubusercontent.com) — no whole-repository download exists for a page
api.github.com/repos/o/r/commits/<ref> works 60/h per IP unauthenticated resolves a branch or tag to a commit
api.github.com/repos/o/r/git/trees/<sha>?recursive=1 works 60/h (same budget) every path, size and mode in one call; not truncated at 3579 files
raw.githubusercontent.com/o/r/<sha>/<path> works none hit fastest — tokio 891 files, 6.1 MB in 4.5 s; pydantic 3579 files, 13.3 MB in 15.8 s (32 requests in flight)
cdn.jsdelivr.net/gh/o/r@<sha>/<path> + data.jsdelivr.com listing works CDN ~5× slower (tokio 22.5 s, pydantic 83.5 s); refuses repositories over 50 MB (llvm-project; all 15 corpus repos are 21–35 MB); listings by branch name are cached and go stale — list at a commit

Two things to handle, both seen in the pydantic run:

  • Transient errors: 46 of 3579 raw.githubusercontent responses were 501 in the browser burst; two repeat bursts of the same 3577 GETs returned all 200. Retry non-200 responses with a short backoff. ~11,000 unauthenticated raw requests in a few minutes drew no 429.
  • Symlinks: jsDelivr returns 404 for one (mode 120000); raw returns the target path as text. Skip them; the validator does not need them.

The one hard limit is the API budget: 60 calls/hour/IP without a token, two per analysis (one if the tree is fetched by branch name and the files by the same name, accepting a seconds-wide race). A shared office address spends it fast. Mitigations: an optional token field (5000/h), jsDelivr as fallback (no API quota, 50 MB cap).

A local folder (<input type="file" webkitdirectory> or the File System Access API) needs no network at all and is the fully private path — the simplest mode to build first.

3. What the validator reads, and the design that follows

Instrumented on the corpus: files() is listed once (cached), size() is called on every file (large-files), exists() on 25–50 configuration paths, and read_text on CI, build and tool configuration plus every source file of the project's languages (facts/testing.py:test_file_facts reads 814 of tokio's 845 files to find tests; settings_facts reads 111). read_head is never called. So lazy per-file fetching would still touch most of the repository, and a synchronous read_text cannot await a fetch. Hence:

  1. Worker: Pyodide in a Web Worker so a 3–25 s run does not freeze the page.
  2. Resolve, list, fetch: resolve o/r@ref to a commit, take the recursive tree, fetch every blob under a size cap (1 MB covers 99.9 % of the corpus files) from raw.githubusercontent.com in parallel with retries, and write each into Pyodide's in-memory FS under /repo. Blobs over the cap are created at their size with truncate so large-files sees them and nothing else reads them; symlinks skipped. jsDelivr as fallback source.
  3. Run: project_validator.cli.main(["/repo", "--format", "json", ...]) with stdout captured; --facts, --answers, --map, --list are already JSON.
  4. Render the JSON; permalink ?repo=o/r@ref.

Per run: the repository's text (6–24 MB for mid-sized corpus projects), 5–16 s in the browser test, once per (repo, commit); Cache-Storage can keep it.

4. Effort and risks

Piece Size
web/index.html + web/worker.js (load Pyodide, fetch, run, render) ~300 lines; the spike is a third of the worker
CI: build the wheel, publish web/ + wheel to Pages (actions/upload-pages-artifact, actions/deploy-pages) one job
Test: fixtures through the wheel under Pyodide in node, compared with native JSON the spike, as a test
Local-folder mode ~40 lines

Risks: the API budget (above); transient raw errors (retry); the Pyodide release tracking 3.14 is new (September 2026) — pin it; URL path encoding (encodeURIComponent per segment); data-heavy repositories download slowly.

The one change worth making in the validator: an entry point that returns the reports as objects instead of printing, so the page does not parse its own stdout.

Suggested order

Local-folder mode first (proves the Pyodide path with no network) → GitHub API + raw.githubusercontent.com with jsDelivr fallback → permalinks and the --facts / --answers / --map views.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt

Linguagem predominante
Python
Estrelas
4
Forks
1
Merge médio
10h 41min
PRs com merge (30d)
79

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de cpp-best-practices/project_validator

Todas as issues de cpp-best-practices/project_validator

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.