Precision and platform: float64 against float32 on CPU, CUDA and Apple Silicon

Aperta
#1 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
20/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python

Direzione di ricerca

Start by reading hardware/benchmark_hardware.py, jax/lax_scan/profile_lax_scan.py, and the RunsOn GPU leg in benchmark.yml; the issue identifies these as starting points for the harness and workloads. Build the harness under precision/ with a README, then collect results as JSON lines under results/. Done means the specified workload and platform measurements, fidelity data, and environment details are recorded, with summary comments on meta#350 and meta#264.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

A harness that measures what float64 costs on each kind of device we build on or own, by kernel type, so that "float64 belongs on the CPU" becomes a measured rule rather than a guess. It runs first on qe-wharton1-gpu.

Why

Three open questions need the same measurements:

  • meta#350, questions 1, 2 and 6: the float64 penalty on the T4, the GPU/CPU crossover, and whether g4dn is the right instance. Its 2026-08-04 comment set out priors and methodology "ahead of building the harness", and the harness has not been built.
  • meta#264, workstream A of meta#335: crossover curves by backend and problem size, "memory bandwidth vs compute bound regimes" included. Not started.
  • project-compute's hardware question: is a large-memory CPU machine (a Mac Studio, or many-core x86) a sensible platform for large float64 JAX models, and does float64 matter to the qe-anu1 card choice? Workstation and GeForce Blackwell cards run FP64 at a small fraction of FP32, so this turns on the same numbers.

Both meta issues carry an offer of qe-wharton1-gpu (two TITAN RTX, the same Turing generation as the T4, plus a 32-core CPU with 128 GB). This issue takes that offer up. hardware/benchmark_hardware.py can't answer the question as it stands: it is float32 only, uses two fixed sizes, and runs JAX on either the GPU or the CPU, never both on the same machine.

A first data point (2026-10-24, an M1 Pro laptop, JAX 0.12.0.1 on CPU): a 4096² float64 matmul runs at 251 GFLOPS under JAX, the T4's FP64 peak (254), and at 441 under NumPy with Accelerate. JAX's CPU backend evidently doesn't use Apple's matrix unit. A Bellman step on a 6000² grid takes 3.2 ms in float64 against 1.7 ms in float32: memory-bound, so twice the bytes means twice the time.

Questions

  1. The float64 penalty, by device and by kernel type: compute-bound, memory-bound and latency-bound.
  2. The crossover: the problem size at which each GPU beats the CPU in the same machine, in float32 and in float64, with compile time counted.
  3. Capacity: for a float64 problem bigger than one card's 24 GB, compare the CPU with 128 GB, two cards sharded over NVLink, and one card recomputing instead of storing.
  4. Apple Silicon as a float64 JAX platform: how M-series CPUs compare with x86 when XLA leaves the matrix unit idle.

The hypothesis: the penalty follows arithmetic intensity. Weak FP64 hurts compute-bound kernels (matmul, solves, transcendentals) and barely touches memory-bound ones, which is the shape of most dynamic-programming lectures' Bellman steps. If that holds, "float64 goes to the CPU" is right for some lectures and wrong for others, and the rule should be set per kernel type, not per precision.

Workloads

Workload Bound by Sizes Why
W1 Dense matmul Compute n = 256 to 8192 The FLOPS ceiling; meta#350's prior is a ~30× penalty
W2a Bellman step, rewards stored: max(R + βV) over choices Memory grid 1k to 60k (60k² in float64 is 28.8 GB, over one card) The usual dynamic-programming lecture shape, and the capacity axis
W2b Bellman step, rewards recomputed from 1-D grids each step (utility of consumption) Compute, transcendental as W2a How you would fit W2a on one card, trading memory for float64 arithmetic, which is exactly what weak-FP64 cards are bad at
W3 lax.scan of small ops (a 10×10 Kalman filter over T steps); reuse jax/lax_scan/profile_lax_scan.py Launch latency T = 1e3 to 1e5 The small-sequential shape meta#350 suspects
W4 As used: the sargent_surico NUTS cell (lecture-python.myst#1022) Mixed as in the lecture Run as a 2 × 2 (CPU/GPU × float32/float64), it separates FP64 from launch latency, which #1022 could not

Platforms

Device Where Phase Why it is here
M1 Pro, 16 GB (CPU) a laptop 0 (first point taken) Apple CPU baseline
M6, 32 GB (CPU) qe-mini1 0 Owned and reachable now
T4 and its host CPU, g4dn.2xlarge RunsOn 0 The CI baseline; benchmark.yml already has a RunsOn GPU leg
Threadripper 2990WX, 32 cores, 128 GB (CPU) qe-wharton1-gpu 1 x86 with large memory
One TITAN RTX: Turing, 24 GB, 672 GB/s, ~0.51 TFLOPS FP64 qe-wharton1-gpu 1 Same generation as the T4, free to rerun
Two TITAN RTX, sharded over the live NVLink bridge qe-wharton1-gpu 1 The capacity question only
M5 Pro, 64 GB (CPU) qe-wharton2, by arrangement with Tom 2 The closest owned machine to a Studio
L4 (g6) and a CPU instance (c7i) RunsOn 2 meta#350's instance-family axis, about USD 1 each
H100, one rented hour cloud 2, needs a yes on spend The float64 ceiling: what a datacentre card buys
RTX PRO 4000 Blackwell qe-anu1 when bought The Blackwell point

Method

The house standard from meta#335 and meta#350:

  • One fresh process per cell (workload × size × precision × device), so compile time counts. Record the cold first call, which includes compile, separately from the warm time: the median of at least five runs after warm-up.
  • Precision is set at process start (JAX_ENABLE_X64) and the device by JAX_PLATFORMS and CUDA_VISIBLE_DEVICES. Assert the result dtype, so a silent float32 can't pass as float64.
  • XLA_PYTHON_CLIENT_PREALLOCATE=false, and nothing else running on the machine.
  • The same JAX release everywhere (0.12.x). On qe-wharton1-gpu that is jax[cuda13], which its driver (595, CUDA 13.2) supports.
  • Fidelity: record the largest absolute and relative difference between the float32 and float64 results in each cell. A speed-up that changes the printed numbers isn't a win for a lecture.
  • Derived figures: achieved GFLOPS (W1), effective GB/s (W2a), the float64/float32 ratio, the GPU/CPU ratio, and the crossover size.
  • Output: one JSON line per cell in results/<host>/<date>.jsonl, each with an environment block (host, CPU, cores, memory, GPU, driver, CUDA, jax and jaxlib versions, flags, commit). Make it a superset of benchmark_hardware.py's system block, and align it with the shared result schema (project-reporting#16) once that exists.

Things on qe-wharton1-gpu that would otherwise spoil the numbers

  • Use the card at 0b:00.0. The one at 43:00.0 drives the desktop. Set CUDA_DEVICE_ORDER=PCI_BUS_ID so the device index means the same thing every run.
  • NUMA. The 2990WX has four NUMA nodes, two of them without local memory. Run the CPU leg twice, with default placement and under numactl --interleave=all, and record numactl --hardware.
  • Memory bandwidth. Quad-channel DDR4-2400 (XMP stays off) caps the CPU at about 77 GB/s on paper, below the M1 Pro's 200. So on memory-bound work the Threadripper is a pessimistic stand-in for "a CPU", which is why the Apple and c7i points matter.
  • Throttling. Log GPU clocks, temperature and power for each cell; two stacked Titans can throttle under sustained load.
  • Run it inside tmux and write results to disk, since the link from Sydney is 236 ms.
  • Run before the GitHub runner is registered, or with it paused. A CI job landing mid-run spoils both.

Phases

  • Phase 0, which needs no Wharton time. Build the harness here, under precision/. Run it on the M1 Pro, qe-mini1 and the RunsOn T4 (by adding a workload input to benchmark.yml). This calibrates the priors and shakes out the harness.
  • Phase 1: qe-wharton1-gpu. CPU, one Titan and two Titans, across the full matrix: about 150 cells, under an hour of machine time. This waits on the machine coming back online (project-compute#72) and on SSH key login (project-compute#73).
  • Phase 2, optional: qe-wharton2, g6 and c7i, and one H100 hour.

What the results would change

If Then
In float64 on W2a, a Titan is well ahead of the CPU "Float64 goes to the CPU" is the wrong blanket rule; pin lectures by kernel type instead. Weak FP64 is then no reason against workstation cards for dynamic-programming work
On W1, W2b or W4, float64 on the GPU is at CPU parity or worse Compute-bound float64 lectures default to the CPU in CI (meta#350, question 1), and the GPU-hours behind the self-hosted runner case shrink
At 60k, one card recomputing (W2b) or two cards sharded keeps up with the CPU Large models don't need large memory, and a Studio's one advantage, capacity, doesn't bind
At 60k, only the CPU finishes in reasonable time Capacity is real, and a large-memory CPU machine becomes a candidate platform
Apple CPUs come close to the x86 CPUs on float64 A Studio is a credible large-memory float64 machine. If they are well behind, a many-core x86 machine is the candidate instead

Deliverables

  • The harness, under precision/, with a README.
  • The results, as JSON lines in results/.
  • A summary comment on meta#350 and meta#264.
  • project-compute writes the hardware consequences up as a dated research file.
Lingua principale
Python
Stelle
0
Fork
1
Merge medio
3h 29m
PR unite (30g)
64

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di QuantEcon/benchmarks

Tutte le issue di QuantEcon/benchmarks

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.