Move perf-benchmark CODE out of pygraphistry into pyg-bench; keep only contract-validated RESULTS for docs

Aperta
  • Qualcosa non va?
#1,827 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Refactoring
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
python
Ambito
performance

Direzione di ricerca

Start by reviewing the 40 files in benchmarks/, especially the named benchmark scripts and filter_pagerank/ and index_* pipelines, alongside manifests/docs_numbers_spec.json and the consumer-side pattern in #1818. Coordinate with pyg-bench’s exporter schema and pygraphistry’s validation before migrating; done means benchmark-generation code has a single source in pyg-bench and docs render only contract-validated results. This is broad follow-on work, not a good standalone first contribution.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

The rule

Performance-benchmark code does not belong in this repo. Only results, for docs.

Owner, 2026-07-28, on merging #1826:

those scripts should live in pyg-bench, generate there & copy into pygraphistry, and then we have data contracts on both sides to ensure right format … perf benchmarks CODE do NOT belong in pygraphistry, just results for docs

Why it matters

A benchmark script living next to shipped product code is measurement machinery with no reproducible environment and no separation from the thing it measures. It invites editing the measurement to fit the claim — and this campaign has already produced three concrete instances of numbers that were wrong for exactly that class of reason:

  • q9's "14.2×" compared GFQL's canonical query against a different Kuzu query (an OPTIONAL MATCH degree-product rewrite). Honest same-query ratio: 1.26×. Fixed in #1826.
  • LadybugDB's "~65×" was a drain-scope figure; at Ladybug's fastest materialization full_scan is 6.18× and two of six cells flip to losses. Fixed in #1826.
  • q8's board win is a cross-call Plottable memo that does not survive ordinary one-shot API use (#1825).

pyg-bench owns generation; pygraphistry renders and re-verifies what it is handed.

Scope

benchmarks/ currently holds 40 files on master, including bench_ladybug_cypher.py (which #1826 just touched), bench_ladybug.py, bench_graphframes.py, the filter_pagerank/ pipelines, and ~15 index_* benchmarks.

Proposed shape

  1. Generation moves to pyg-bench. Scripts are ported, not copied — the pygraphistry copy is deleted so there is one source of truth.
  2. Results cross the boundary as data, with a contract on both sides: an exporter schema in pyg-bench and consumer validation in pygraphistry. The manifests/docs_numbers_spec.json + #1818 mechanism is the existing pattern to extend; the Ladybug cells are already flagged as not yet exported through it.
  3. Docs render only validated results — a figure with no contract-validated provenance does not get published.
  4. benchmarks/ becomes legacy: nothing new lands there while migration proceeds.

Not a blocker

This is a follow-on. In-flight PRs are unaffected and should not be held for it.

Related

  • #1818 — consumer side of the numbers contract (unmerged)
  • #1825, #1824 — defects found by this campaign's measurement work
  • pyg-bench #116 — the q9/Ladybug generation side, already merged there
Lingua principale
Python
Stelle
2.6k
Fork
229
Merge medio
1g 5h
PR unite (30g)
99

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di graphistry/pygraphistry

Tutte le issue di graphistry/pygraphistry

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.