Run the validator in the browser from GitHub Pages, pointed at a public repository (Pyodide)

Ouverte
  • Quelque chose ne va pas ?
#127 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
35/100
Type d'issue
Fonctionnalité
Clarté
Clairement spécifiée
Activité
Active
Stack technique
javascript, python
Domaine
devtools, web-dev

Piste de recherche

Start with the Pyodide spike described in the issue and the validator CLI entry point, then build the local-folder flow in web/index.html and web/worker.js. The proposal gives a staged path through GitHub API and raw file fetching, rendering, permalinks, and CI deployment to GitHub Pages. Done means the browser can validate a local folder and public repository and display the validator's JSON output.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Proposal

A static page on GitHub Pages that runs the validator in the visitor's browser (no server), pointed at any public GitHub repository (?repo=owner/name@ref) or at a local folder. Feasibility was investigated on 2026-11-08 against rearchitecture (#126, head 9d27689); findings below. The validator itself needs no change.

1. The validator runs unmodified under Pyodide

Pyodide is CPython compiled to WebAssembly. [email protected] ships Python 3.14.2 (the package version now tracks Python's), so requires-python >= 3.14 and the 3.14-only syntax in the code base (except ValueError, re.error:) are fine, and it ships a wasm wheel of PyYAML 6.0.3, the one dependency.

Spike: build the wheel (pip wheel . → project_validator-0.1.0-py3-none-any.whl, 340 KB, pure Python), load Pyodide in node, micropip.install the wheel, mount a checkout, run python -m project_validator <dir> --format json with stdout captured.

Repository Files Native Pyodide Output
cons_expr 73 0.7 s 2.4 s byte-identical JSON
tokio 874 2.7 s 8.5 s byte-identical JSON
prometheus 1742 8.8 s 24.4 s byte-identical JSON

About 3× native. One-time runtime load is ~13 MB (pyodide.asm.wasm 9.2 MB, stdlib 2.5 MB, PyYAML, the wheel), browser-cached afterwards; Pyodide is on the jsDelivr CDN so Pages hosts only the page and the wheel.

  • git is absent: shutil.which("git") is None, so Project.tracked_files() falls back to every file. For a snapshot of a repository that is the correct answer (a snapshot has no untracked files), the same as validating a tarball.
  • --build (cmake/ctest through subprocess) cannot work in a browser and would be hidden. Everything else is static reading.
Spike script (node) 
import { loadPyodide } from "pyodide";
const [,, wheel, target, ...args] = process.argv;
const pyodide = await loadPyodide();
await pyodide.loadPackage(["pyyaml", "micropip"]);
pyodide.FS.mkdirTree("/wheels");
pyodide.mountNodeFS("/wheels", wheel.slice(0, wheel.lastIndexOf("/")));
await pyodide.pyimport("micropip").install("emfs:/wheels/" + wheel.slice(wheel.lastIndexOf("/") + 1));
pyodide.FS.mkdirTree("/proj");
pyodide.mountNodeFS("/proj", target);
pyodide.runPython(`
import sys, runpy, io
sys.argv = ["project-validator", "/proj", *${JSON.stringify(args)}]
out = io.StringIO(); sys.stdout = out
try:
    runpy.run_module("project_validator", run_name="__main__")
except SystemExit as e:
    code = e.code
sys.stdout = sys.__stdout__
open("/wheels/out.json", "w").write(out.getvalue())
`);

2. Getting a repository into the page — verified in a real browser

Tested from a page served on http://127.0.0.1:8765 (a different origin from GitHub, as a Pages site is) in headless Firefox, fetching cons_expr, tokio and pydantic. Browsers enforce CORS; curl does not, so this is the test that counts.

Source From the page Rate limit Note
api.github.com/repos/o/r/tarball, codeload.github.com/…/tar.gz, github.com/…/archive/….tar.gz fails (NetworkError; codeload's CORS allows only render.githubusercontent.com) — no whole-repository download exists for a page
api.github.com/repos/o/r/commits/<ref> works 60/h per IP unauthenticated resolves a branch or tag to a commit
api.github.com/repos/o/r/git/trees/<sha>?recursive=1 works 60/h (same budget) every path, size and mode in one call; not truncated at 3579 files
raw.githubusercontent.com/o/r/<sha>/<path> works none hit fastest — tokio 891 files, 6.1 MB in 4.5 s; pydantic 3579 files, 13.3 MB in 15.8 s (32 requests in flight)
cdn.jsdelivr.net/gh/o/r@<sha>/<path> + data.jsdelivr.com listing works CDN ~5× slower (tokio 22.5 s, pydantic 83.5 s); refuses repositories over 50 MB (llvm-project; all 15 corpus repos are 21–35 MB); listings by branch name are cached and go stale — list at a commit

Two things to handle, both seen in the pydantic run:

  • Transient errors: 46 of 3579 raw.githubusercontent responses were 501 in the browser burst; two repeat bursts of the same 3577 GETs returned all 200. Retry non-200 responses with a short backoff. ~11,000 unauthenticated raw requests in a few minutes drew no 429.
  • Symlinks: jsDelivr returns 404 for one (mode 120000); raw returns the target path as text. Skip them; the validator does not need them.

The one hard limit is the API budget: 60 calls/hour/IP without a token, two per analysis (one if the tree is fetched by branch name and the files by the same name, accepting a seconds-wide race). A shared office address spends it fast. Mitigations: an optional token field (5000/h), jsDelivr as fallback (no API quota, 50 MB cap).

A local folder (<input type="file" webkitdirectory> or the File System Access API) needs no network at all and is the fully private path — the simplest mode to build first.

3. What the validator reads, and the design that follows

Instrumented on the corpus: files() is listed once (cached), size() is called on every file (large-files), exists() on 25–50 configuration paths, and read_text on CI, build and tool configuration plus every source file of the project's languages (facts/testing.py:test_file_facts reads 814 of tokio's 845 files to find tests; settings_facts reads 111). read_head is never called. So lazy per-file fetching would still touch most of the repository, and a synchronous read_text cannot await a fetch. Hence:

  1. Worker: Pyodide in a Web Worker so a 3–25 s run does not freeze the page.
  2. Resolve, list, fetch: resolve o/r@ref to a commit, take the recursive tree, fetch every blob under a size cap (1 MB covers 99.9 % of the corpus files) from raw.githubusercontent.com in parallel with retries, and write each into Pyodide's in-memory FS under /repo. Blobs over the cap are created at their size with truncate so large-files sees them and nothing else reads them; symlinks skipped. jsDelivr as fallback source.
  3. Run: project_validator.cli.main(["/repo", "--format", "json", ...]) with stdout captured; --facts, --answers, --map, --list are already JSON.
  4. Render the JSON; permalink ?repo=o/r@ref.

Per run: the repository's text (6–24 MB for mid-sized corpus projects), 5–16 s in the browser test, once per (repo, commit); Cache-Storage can keep it.

4. Effort and risks

Piece Size
web/index.html + web/worker.js (load Pyodide, fetch, run, render) ~300 lines; the spike is a third of the worker
CI: build the wheel, publish web/ + wheel to Pages (actions/upload-pages-artifact, actions/deploy-pages) one job
Test: fixtures through the wheel under Pyodide in node, compared with native JSON the spike, as a test
Local-folder mode ~40 lines

Risks: the API budget (above); transient raw errors (retry); the Pyodide release tracking 3.14 is new (September 2026) — pin it; URL path encoding (encodeURIComponent per segment); data-heavy repositories download slowly.

The one change worth making in the validator: an entry point that returns the reports as objects instead of printing, so the page does not parse its own stdout.

Suggested order

Local-folder mode first (proves the Pyodide path with no network) → GitHub API + raw.githubusercontent.com with jsDelivr fallback → permalinks and the --facts / --answers / --map views.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt

Langage dominant
Python
Étoiles
4
Forks
1
Merge moyen
10 h 41 min
PR mergées (30 j)
79

Préparer son environnement

Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de cpp-best-practices/project_validator

Toutes les issues de cpp-best-practices/project_validator

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.