Precision and platform: float64 against float32 on CPU, CUDA and Apple Silicon
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 20/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- python
- Ambito
- machine-learning, performance
Direzione di ricerca
Start by reading hardware/benchmark_hardware.py, jax/lax_scan/profile_lax_scan.py, and the RunsOn GPU leg in benchmark.yml; the issue identifies these as starting points for the harness and workloads. Build the harness under precision/ with a README, then collect results as JSON lines under results/. Done means the specified workload and platform measurements, fidelity data, and environment details are recorded, with summary comments on meta#350 and meta#264.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
A harness that measures what float64 costs on each kind of device we build on or own, by kernel type, so that "float64 belongs on the CPU" becomes a measured rule rather than a guess. It runs first on qe-wharton1-gpu.
Why
Three open questions need the same measurements:
- meta#350, questions 1, 2 and 6: the float64 penalty on the T4, the GPU/CPU crossover, and whether
g4dnis the right instance. Its 2026-08-04 comment set out priors and methodology "ahead of building the harness", and the harness has not been built. - meta#264, workstream A of meta#335: crossover curves by backend and problem size, "memory bandwidth vs compute bound regimes" included. Not started.
- project-compute's hardware question: is a large-memory CPU machine (a Mac Studio, or many-core x86) a sensible platform for large float64 JAX models, and does float64 matter to the
qe-anu1card choice? Workstation and GeForce Blackwell cards run FP64 at a small fraction of FP32, so this turns on the same numbers.
Both meta issues carry an offer of qe-wharton1-gpu (two TITAN RTX, the same Turing generation as the T4, plus a 32-core CPU with 128 GB). This issue takes that offer up. hardware/benchmark_hardware.py can't answer the question as it stands: it is float32 only, uses two fixed sizes, and runs JAX on either the GPU or the CPU, never both on the same machine.
A first data point (2026-10-24, an M1 Pro laptop, JAX 0.12.0.1 on CPU): a 4096² float64 matmul runs at 251 GFLOPS under JAX, the T4's FP64 peak (254), and at 441 under NumPy with Accelerate. JAX's CPU backend evidently doesn't use Apple's matrix unit. A Bellman step on a 6000² grid takes 3.2 ms in float64 against 1.7 ms in float32: memory-bound, so twice the bytes means twice the time.
Questions
- The float64 penalty, by device and by kernel type: compute-bound, memory-bound and latency-bound.
- The crossover: the problem size at which each GPU beats the CPU in the same machine, in float32 and in float64, with compile time counted.
- Capacity: for a float64 problem bigger than one card's 24 GB, compare the CPU with 128 GB, two cards sharded over NVLink, and one card recomputing instead of storing.
- Apple Silicon as a float64 JAX platform: how M-series CPUs compare with x86 when XLA leaves the matrix unit idle.
The hypothesis: the penalty follows arithmetic intensity. Weak FP64 hurts compute-bound kernels (matmul, solves, transcendentals) and barely touches memory-bound ones, which is the shape of most dynamic-programming lectures' Bellman steps. If that holds, "float64 goes to the CPU" is right for some lectures and wrong for others, and the rule should be set per kernel type, not per precision.
Workloads
| Workload | Bound by | Sizes | Why | |
|---|---|---|---|---|
| W1 | Dense matmul | Compute | n = 256 to 8192 | The FLOPS ceiling; meta#350's prior is a ~30× penalty |
| W2a | Bellman step, rewards stored: max(R + βV) over choices |
Memory | grid 1k to 60k (60k² in float64 is 28.8 GB, over one card) | The usual dynamic-programming lecture shape, and the capacity axis |
| W2b | Bellman step, rewards recomputed from 1-D grids each step (utility of consumption) | Compute, transcendental | as W2a | How you would fit W2a on one card, trading memory for float64 arithmetic, which is exactly what weak-FP64 cards are bad at |
| W3 | lax.scan of small ops (a 10×10 Kalman filter over T steps); reuse jax/lax_scan/profile_lax_scan.py |
Launch latency | T = 1e3 to 1e5 | The small-sequential shape meta#350 suspects |
| W4 | As used: the sargent_surico NUTS cell (lecture-python.myst#1022) |
Mixed | as in the lecture | Run as a 2 × 2 (CPU/GPU × float32/float64), it separates FP64 from launch latency, which #1022 could not |
Platforms
| Device | Where | Phase | Why it is here |
|---|---|---|---|
| M1 Pro, 16 GB (CPU) | a laptop | 0 (first point taken) | Apple CPU baseline |
| M6, 32 GB (CPU) | qe-mini1 |
0 | Owned and reachable now |
T4 and its host CPU, g4dn.2xlarge |
RunsOn | 0 | The CI baseline; benchmark.yml already has a RunsOn GPU leg |
| Threadripper 2990WX, 32 cores, 128 GB (CPU) | qe-wharton1-gpu |
1 | x86 with large memory |
| One TITAN RTX: Turing, 24 GB, 672 GB/s, ~0.51 TFLOPS FP64 | qe-wharton1-gpu |
1 | Same generation as the T4, free to rerun |
| Two TITAN RTX, sharded over the live NVLink bridge | qe-wharton1-gpu |
1 | The capacity question only |
| M5 Pro, 64 GB (CPU) | qe-wharton2, by arrangement with Tom |
2 | The closest owned machine to a Studio |
L4 (g6) and a CPU instance (c7i) |
RunsOn | 2 | meta#350's instance-family axis, about USD 1 each |
| H100, one rented hour | cloud | 2, needs a yes on spend | The float64 ceiling: what a datacentre card buys |
| RTX PRO 4000 Blackwell | qe-anu1 |
when bought | The Blackwell point |
Method
The house standard from meta#335 and meta#350:
- One fresh process per cell (workload × size × precision × device), so compile time counts. Record the cold first call, which includes compile, separately from the warm time: the median of at least five runs after warm-up.
- Precision is set at process start (
JAX_ENABLE_X64) and the device byJAX_PLATFORMSandCUDA_VISIBLE_DEVICES. Assert the result dtype, so a silent float32 can't pass as float64. XLA_PYTHON_CLIENT_PREALLOCATE=false, and nothing else running on the machine.- The same JAX release everywhere (0.12.x). On
qe-wharton1-gputhat isjax[cuda13], which its driver (595, CUDA 13.2) supports. - Fidelity: record the largest absolute and relative difference between the float32 and float64 results in each cell. A speed-up that changes the printed numbers isn't a win for a lecture.
- Derived figures: achieved GFLOPS (W1), effective GB/s (W2a), the float64/float32 ratio, the GPU/CPU ratio, and the crossover size.
- Output: one JSON line per cell in
results/<host>/<date>.jsonl, each with an environment block (host, CPU, cores, memory, GPU, driver, CUDA,jaxandjaxlibversions, flags, commit). Make it a superset ofbenchmark_hardware.py'ssystemblock, and align it with the shared result schema (project-reporting#16) once that exists.
Things on qe-wharton1-gpu that would otherwise spoil the numbers
- Use the card at
0b:00.0. The one at43:00.0drives the desktop. SetCUDA_DEVICE_ORDER=PCI_BUS_IDso the device index means the same thing every run. - NUMA. The 2990WX has four NUMA nodes, two of them without local memory. Run the CPU leg twice, with default placement and under
numactl --interleave=all, and recordnumactl --hardware. - Memory bandwidth. Quad-channel DDR4-2400 (XMP stays off) caps the CPU at about 77 GB/s on paper, below the M1 Pro's 200. So on memory-bound work the Threadripper is a pessimistic stand-in for "a CPU", which is why the Apple and
c7ipoints matter. - Throttling. Log GPU clocks, temperature and power for each cell; two stacked Titans can throttle under sustained load.
- Run it inside
tmuxand write results to disk, since the link from Sydney is 236 ms. - Run before the GitHub runner is registered, or with it paused. A CI job landing mid-run spoils both.
Phases
- Phase 0, which needs no Wharton time. Build the harness here, under
precision/. Run it on the M1 Pro,qe-mini1and the RunsOn T4 (by adding a workload input tobenchmark.yml). This calibrates the priors and shakes out the harness. - Phase 1:
qe-wharton1-gpu. CPU, one Titan and two Titans, across the full matrix: about 150 cells, under an hour of machine time. This waits on the machine coming back online (project-compute#72) and on SSH key login (project-compute#73). - Phase 2, optional:
qe-wharton2,g6andc7i, and one H100 hour.
What the results would change
| If | Then |
|---|---|
| In float64 on W2a, a Titan is well ahead of the CPU | "Float64 goes to the CPU" is the wrong blanket rule; pin lectures by kernel type instead. Weak FP64 is then no reason against workstation cards for dynamic-programming work |
| On W1, W2b or W4, float64 on the GPU is at CPU parity or worse | Compute-bound float64 lectures default to the CPU in CI (meta#350, question 1), and the GPU-hours behind the self-hosted runner case shrink |
| At 60k, one card recomputing (W2b) or two cards sharded keeps up with the CPU | Large models don't need large memory, and a Studio's one advantage, capacity, doesn't bind |
| At 60k, only the CPU finishes in reasonable time | Capacity is real, and a large-memory CPU machine becomes a candidate platform |
| Apple CPUs come close to the x86 CPUs on float64 | A Studio is a credible large-memory float64 machine. If they are well behind, a many-core x86 machine is the candidate instead |
Deliverables
- The harness, under
precision/, with a README. - The results, as JSON lines in
results/. - A summary comment on meta#350 and meta#264.
- project-compute writes the hardware consequences up as a dated research file.
- Lingua principale
- Python
- Stelle
- 0
- Fork
- 1
- Merge medio
- 3h 29m
- PR unite (30g)
- 64
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di QuantEcon/benchmarks
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
QuantEcon/benchmarks#78 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 25/100
QuantEcon/benchmarks#72 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
P5: Markov chains, starting with the stationary distribution of a savings policy (aiyagari_jax)Aperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 45/100
QuantEcon/benchmarks#47 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
QuantEcon/benchmarks#27 · 2 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
Float32 fidelity: which float64 lectures need float64, and can the heaviest step drop to float32?Aperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
QuantEcon/benchmarks#11 · 2 commenti ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di QuantEcon/benchmarks
Issue simili
-
needs-triage
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 88/100
flashinfer-ai/flashinfer#9803 ·
I maintainer di solito rispondono entro 1 giorno
-
bug frontend good first issue help wanted
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
oscarbol09/thesisforge#61 ·
-
api bug performance
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
oscarbol09/audiobard#159 ·
-
area:web p2 product:business triage:active type:bug
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 89/100
cuadrao-ai/argus#2328 ·
I maintainer di solito rispondono entro 1 giorno
-
bug frontend good first issue help wanted
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
nikhilb2/simple_invoicing#907 ·