Performance: contour PNG export misses the 10× Matplotlib target (8.50× in release audit)

Open
  • Is something wrong?
#552 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
20/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
matplotlib, python, rust
Domain
data-visualization, performance, testing-qa

Research direction

Start with benchmarks/bench_pyplot_vs_matplotlib.py and spec/benchmarks/methodology.md at the audited commit; first correct the XY save call to export at 200 DPI, then confirm the contour gap on a controlled runner and record raw samples, dimensions, and tier decisions. Profile build, rasterization, and PNG encoding separately before choosing an optimization. Done means meeting the contour 10× standard target, or documenting reproducible evidence for revising it, while preserving output correctness and adding benchmark coverage.

Written by the indexing model from the issue text.

Description

The matched-output release benchmark found a contour export gap against the existing per-family 10× Matplotlib/Agg goal in the benchmark methodology.

Workload XY total median Matplotlib total median Speedup XY tier
80 x 100 / 8 levels 10.99 ms 93.38 ms 8.50× direct

Both outputs are nonblank 1800×840 PNGs. The metric is figure construction through a completed compressed PNG, excluding imports, data generation and warm-up. XY renders this row directly. Preserve contour topology, level placement and labeling behavior.

These are diagnostic results from three repetitions on a shared workstation, with one warm-up, recorded during the 2026-11-05 release audit. Confirm the gap on a controlled runner before selecting an optimization. This does not establish a regression from v0.0.8.

Reproduce

Audited source: 5e58b12c0a1b89a58c4555af370204ef1ed0515e plus local release-validation fixes. Environment: Linux x86_64, 12 logical CPUs, CPython 3.14.6, NumPy 2.5.1, Matplotlib 3.11.0, native XY core, Rust 1.96.0.

Harness prerequisite: the audited upstream bench_pyplot_vs_matplotlib.py omits export DPI in _xy_render. Set its save call to the following before measuring, matching the Matplotlib adapter:

fig.savefig(out, format="png", dpi=EXPORT_DPI)

Both adapters build at 100 DPI and export at 200 DPI. Without that correction, the harness fails its dimension check before producing a valid comparison. The correction and output-size regression test are prepared locally on farhan/release-validation, but have not landed upstream.

cargo build --release
uv sync --extra matplotlib --group dev
uv pip install 'matplotlib==3.11.0'
MPLBACKEND=Agg uv run --no-sync python benchmarks/bench_pyplot_vs_matplotlib.py \
  --profile smoke --reps 3 --warmups 1 \
  --json /tmp/pyplot-smoke.json

The harness runs all six families; inspect the contour row. For confirmation and the existing standard target gate:

MPLBACKEND=Agg uv run --no-sync python benchmarks/bench_pyplot_vs_matplotlib.py \
  --profile standard --reps 21 --warmups 3 \
  --target-speedup 10 --require-target \
  --json /tmp/pyplot-standard.json

--require-target gates every family, so a failure in another family does not by itself establish this issue's outcome.

Raw observations
[
  {
    "library": "xy.pyplot",
    "total_samples_ms": [
      10.909707,
      10.98602,
      11.471173
    ],
    "build_median_ms": 0.357385,
    "render_median_ms": 10.587838,
    "total_median_ms": 10.98602,
    "total_p95_ms": 11.471173,
    "png_bytes": 314653
  },
  {
    "library": "matplotlib",
    "total_samples_ms": [
      93.381723,
      92.888934,
      95.391998
    ],
    "build_median_ms": 5.579867,
    "render_median_ms": 87.794813,
    "total_median_ms": 93.38172300000001,
    "total_p95_ms": 95.391998,
    "png_bytes": 239043
  }
]
Acceptance
  • Reproduce on a controlled runner and record environment, exact commit, raw samples, output dimensions and tier decisions.
  • Profile build, rasterization and PNG encoding separately; optimize the measured bottleneck.
  • Meet the existing standard-profile 10× total-time target for contour, or explicitly revise the specification with reproducible evidence explaining why the target is not met.
  • Retain image correctness, labels and dimensions; do not achieve the target by silently reducing visual quality or dropping data.
  • Record smoke, standard and huge profile results and output sizes, and add a meaningful regression benchmark.
  • Update benchmark evidence and specifications with the resulting behavior.

The local audit report and complete raw comparison are prepared under spec/process/release-validation/2026-11-05/; they are currently uncommitted. The observations above are included here so this issue is reproducible without access to that worktree.

Dominant language
Python
Stars
1.9k
Forks
79
Avg merge
1d 39m
Merged PRs (30d)
22

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from reflex-dev/xy

All issues in reflex-dev/xy

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.