Run the validator in the browser from GitHub Pages, pointed at a public repository (Pyodide)

Đang mở
  • Có gì không ổn?
#127 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
35/100
Loại issue
Tính năng
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
javascript, python
Lĩnh vực
devtools, web-dev

Hướng nghiên cứu

Start with the Pyodide spike described in the issue and the validator CLI entry point, then build the local-folder flow in web/index.html and web/worker.js. The proposal gives a staged path through GitHub API and raw file fetching, rendering, permalinks, and CI deployment to GitHub Pages. Done means the browser can validate a local folder and public repository and display the validator's JSON output.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Proposal

A static page on GitHub Pages that runs the validator in the visitor's browser (no server), pointed at any public GitHub repository (?repo=owner/name@ref) or at a local folder. Feasibility was investigated on 2026-11-08 against rearchitecture (#126, head 9d27689); findings below. The validator itself needs no change.

1. The validator runs unmodified under Pyodide

Pyodide is CPython compiled to WebAssembly. [email protected] ships Python 3.14.2 (the package version now tracks Python's), so requires-python >= 3.14 and the 3.14-only syntax in the code base (except ValueError, re.error:) are fine, and it ships a wasm wheel of PyYAML 6.0.3, the one dependency.

Spike: build the wheel (pip wheel . → project_validator-0.1.0-py3-none-any.whl, 340 KB, pure Python), load Pyodide in node, micropip.install the wheel, mount a checkout, run python -m project_validator <dir> --format json with stdout captured.

Repository Files Native Pyodide Output
cons_expr 73 0.7 s 2.4 s byte-identical JSON
tokio 874 2.7 s 8.5 s byte-identical JSON
prometheus 1742 8.8 s 24.4 s byte-identical JSON

About 3× native. One-time runtime load is ~13 MB (pyodide.asm.wasm 9.2 MB, stdlib 2.5 MB, PyYAML, the wheel), browser-cached afterwards; Pyodide is on the jsDelivr CDN so Pages hosts only the page and the wheel.

  • git is absent: shutil.which("git") is None, so Project.tracked_files() falls back to every file. For a snapshot of a repository that is the correct answer (a snapshot has no untracked files), the same as validating a tarball.
  • --build (cmake/ctest through subprocess) cannot work in a browser and would be hidden. Everything else is static reading.
Spike script (node) 
import { loadPyodide } from "pyodide";
const [,, wheel, target, ...args] = process.argv;
const pyodide = await loadPyodide();
await pyodide.loadPackage(["pyyaml", "micropip"]);
pyodide.FS.mkdirTree("/wheels");
pyodide.mountNodeFS("/wheels", wheel.slice(0, wheel.lastIndexOf("/")));
await pyodide.pyimport("micropip").install("emfs:/wheels/" + wheel.slice(wheel.lastIndexOf("/") + 1));
pyodide.FS.mkdirTree("/proj");
pyodide.mountNodeFS("/proj", target);
pyodide.runPython(`
import sys, runpy, io
sys.argv = ["project-validator", "/proj", *${JSON.stringify(args)}]
out = io.StringIO(); sys.stdout = out
try:
    runpy.run_module("project_validator", run_name="__main__")
except SystemExit as e:
    code = e.code
sys.stdout = sys.__stdout__
open("/wheels/out.json", "w").write(out.getvalue())
`);

2. Getting a repository into the page — verified in a real browser

Tested from a page served on http://127.0.0.1:8765 (a different origin from GitHub, as a Pages site is) in headless Firefox, fetching cons_expr, tokio and pydantic. Browsers enforce CORS; curl does not, so this is the test that counts.

Source From the page Rate limit Note
api.github.com/repos/o/r/tarball, codeload.github.com/…/tar.gz, github.com/…/archive/….tar.gz fails (NetworkError; codeload's CORS allows only render.githubusercontent.com) — no whole-repository download exists for a page
api.github.com/repos/o/r/commits/<ref> works 60/h per IP unauthenticated resolves a branch or tag to a commit
api.github.com/repos/o/r/git/trees/<sha>?recursive=1 works 60/h (same budget) every path, size and mode in one call; not truncated at 3579 files
raw.githubusercontent.com/o/r/<sha>/<path> works none hit fastest — tokio 891 files, 6.1 MB in 4.5 s; pydantic 3579 files, 13.3 MB in 15.8 s (32 requests in flight)
cdn.jsdelivr.net/gh/o/r@<sha>/<path> + data.jsdelivr.com listing works CDN ~5× slower (tokio 22.5 s, pydantic 83.5 s); refuses repositories over 50 MB (llvm-project; all 15 corpus repos are 21–35 MB); listings by branch name are cached and go stale — list at a commit

Two things to handle, both seen in the pydantic run:

  • Transient errors: 46 of 3579 raw.githubusercontent responses were 501 in the browser burst; two repeat bursts of the same 3577 GETs returned all 200. Retry non-200 responses with a short backoff. ~11,000 unauthenticated raw requests in a few minutes drew no 429.
  • Symlinks: jsDelivr returns 404 for one (mode 120000); raw returns the target path as text. Skip them; the validator does not need them.

The one hard limit is the API budget: 60 calls/hour/IP without a token, two per analysis (one if the tree is fetched by branch name and the files by the same name, accepting a seconds-wide race). A shared office address spends it fast. Mitigations: an optional token field (5000/h), jsDelivr as fallback (no API quota, 50 MB cap).

A local folder (<input type="file" webkitdirectory> or the File System Access API) needs no network at all and is the fully private path — the simplest mode to build first.

3. What the validator reads, and the design that follows

Instrumented on the corpus: files() is listed once (cached), size() is called on every file (large-files), exists() on 25–50 configuration paths, and read_text on CI, build and tool configuration plus every source file of the project's languages (facts/testing.py:test_file_facts reads 814 of tokio's 845 files to find tests; settings_facts reads 111). read_head is never called. So lazy per-file fetching would still touch most of the repository, and a synchronous read_text cannot await a fetch. Hence:

  1. Worker: Pyodide in a Web Worker so a 3–25 s run does not freeze the page.
  2. Resolve, list, fetch: resolve o/r@ref to a commit, take the recursive tree, fetch every blob under a size cap (1 MB covers 99.9 % of the corpus files) from raw.githubusercontent.com in parallel with retries, and write each into Pyodide's in-memory FS under /repo. Blobs over the cap are created at their size with truncate so large-files sees them and nothing else reads them; symlinks skipped. jsDelivr as fallback source.
  3. Run: project_validator.cli.main(["/repo", "--format", "json", ...]) with stdout captured; --facts, --answers, --map, --list are already JSON.
  4. Render the JSON; permalink ?repo=o/r@ref.

Per run: the repository's text (6–24 MB for mid-sized corpus projects), 5–16 s in the browser test, once per (repo, commit); Cache-Storage can keep it.

4. Effort and risks

Piece Size
web/index.html + web/worker.js (load Pyodide, fetch, run, render) ~300 lines; the spike is a third of the worker
CI: build the wheel, publish web/ + wheel to Pages (actions/upload-pages-artifact, actions/deploy-pages) one job
Test: fixtures through the wheel under Pyodide in node, compared with native JSON the spike, as a test
Local-folder mode ~40 lines

Risks: the API budget (above); transient raw errors (retry); the Pyodide release tracking 3.14 is new (September 2026) — pin it; URL path encoding (encodeURIComponent per segment); data-heavy repositories download slowly.

The one change worth making in the validator: an entry point that returns the reports as objects instead of printing, so the page does not parse its own stdout.

Suggested order

Local-folder mode first (proves the Pyodide path with no network) → GitHub API + raw.githubusercontent.com with jsDelivr fallback → permalinks and the --facts / --answers / --map views.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt

Ngôn ngữ chính
Python
Star
4
Fork
1
Merge trung bình
10 giờ 41 phút
Pull request đã merge (30 ngày)
79

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của cpp-best-practices/project_validator

Tất cả issue của cpp-best-practices/project_validator

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.