SUGGESTION: Run the OpenCL tests under Oclgrind in CI to catch data races and invalid memory accesses in kernels

Aperta
  • Qualcosa non va?
#3,430 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@SteveBronder ci sta già lavorando.

Dal 5/10/2026.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
ci-cd, testing

Direzione di ricerca

Start by reading how runTests.py prefixes MPI tests, then inspect the CI setup for where a CPU-only OpenCL test stage belongs. Build test/unit/math/opencl with STAN_OPENCL=true and run its test binaries through Oclgrind with race logging. Done means the stage fails on non-empty logs and can later be made required once the reported races are fixed.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

code cleanup continuous integration dependencies gpu

Description

Motivation. CI runs the OpenCL tests on one GPU. A kernel bug that this device happens to tolerate stays invisible. Example: https://github.com/stan-dev/math/issues/3429, where the cholesky_decompose kernel returns a wrong factor on an AMD GPU because its barriers fence local memory only.

Oclgrind is an OpenCL device simulator. It needs no GPU and checks what real devices do not:

  • data races between work items, including a barrier with the wrong fence flag (--data-races),
  • reads and writes outside a buffer (always on),
  • use of uninitialized values (--uninitialized).

It runs our test binaries unchanged:

oclgrind --data-races --log races.log ./test/unit/math/opencl/cholesky_decompose_test

On develop this reports read-write races in cholesky_decompose, already for the 3×3 case, and nothing once the barriers are fixed. A first local pass also reports races in tridiagonalization_householder.

Suggested steps.

  1. Add oclgrind to the CI image.
  2. Add a CPU-only stage that builds test/unit/math/opencl with STAN_OPENCL=true and runs each test binary through oclgrind --data-races --log <file>. runTests.py already prefixes MPI tests with mpirun; the same mechanism can prefix oclgrind.
  3. Fail the stage when a log is not empty.
  4. Start it as non-blocking and make it required once the existing reports are fixed.
Lingua principale
C++
Stelle
845
Fork
225
Merge medio
3g 9h
PR unite (30g)
12

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di stan-dev/math

Tutte le issue di stan-dev/math

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.