onnxruntime CoreML execution provider leaks ~70-78 KB per inference on macOS (RF-DETR default config)

Aperta
  • Qualcosa non va?
#3,302 13 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@Nishant-ZFYII ci sta già lavorando.

Dal 30/10/2026.

  • #3402 di @davidnichols-ops — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
macos, python
Ambito
performance

Direzione di ricerca

Start by reading inference_models/inference_models/models/common/onnx.py, especially get_default_coreml_provider_options, to understand the CoreML options used by default. The standalone reproduction script is on the reporter’s machine and can be attached on request; first ask for it, then compare memory growth with CoreML and CPU providers. Done means the native memory growth per inference is explained and a fix or agreed mitigation is established.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

On macOS, running an ONNX model through onnxruntime's CoreMLExecutionProvider with the options inference_models sets by default for RF-DETR (ModelFormat=MLProgram, MLComputeUnits=CPUAndGPU, see inference_models/inference_models/models/common/onnx.py, get_default_coreml_provider_options) grows process memory by roughly 70-78 KB per session.run call. At 30 inferences per second that is about 1.9-2.0 MB/s, or 6-7 GB per day. The leak is native (not in the Python heap) and is present without any Roboflow code in the loop.

Found while validating an unrelated Workflows feature; it is not caused by that work.

Evidence

Mac M4 Max, macOS 27.0.1, Python 3.10, rfdetr-nano (coco/40 weights). Memory is phys_footprint sampled every second; slopes are linear fits for t >= 30 s.

Run Growth
InferencePipeline + RF-DETR + visualizations, pristine main (18eb2936e) +1.95 MB/s, R² 0.98
Model only through the workflow engine +1.90 MB/s, R² 0.99
Model only with ONNXRUNTIME_EXECUTION_PROVIDERS=CPUExecutionProvider flat (-0.18 MB/s)
Passthrough workflow, no model flat
Bare onnxruntime 1.21.1 loop, CoreML MLProgram CPUAndGPU, no Roboflow code +1.43 MB/s at 18.4 it/s = 78 KB per call, R² 1.00
Same loop, onnxruntime 1.31.1 (latest) +1.96 to 2.15 MB/s at 28.7 it/s
CoreML MLProgram, MLComputeUnits=CPUOnly flat, 14.8 it/s
CoreML NeuralNetwork format flat, but 3 it/s and a 3.7 GB footprint
Objective-C autorelease pool pushed/popped around each run still +2.35 MB/s

tracemalloc shows the Python heap shrinking while RSS grows (94.8 to 84.3 MB traced vs +180 MB process over the same 180 s), so nothing Python-side accumulates. The leak is in the MLProgram GPU path.

Repro

A standalone script that loads the cached rfdetr-nano weights with the same CoreML options and runs a paced loop while sampling memory lives at ~/Code/vision-event-clips/scripts/g1.9/ort_loop.py on Brad's machine (can be attached on request):

python ort_loop.py --provider coreml --seconds 300 --fps 30 --label coreml   # leaks
python ort_loop.py --provider cpu    --seconds 300 --fps 30 --label cpu      # flat

Impact

Any long-running RF-DETR (or other MLProgram-opted-in model) deployment on macOS with the default provider options, e.g. InferencePipeline on a Mac for hours.

Mitigation candidates

  • MLComputeUnits=CPUOnly for CoreML: no leak, about half the throughput (14.8 vs 30 it/s on this model).
  • Periodic InferenceSession recycling in the ONNX model wrapper.
  • Upstream report to onnxruntime with the bare-loop repro (1.21.1 and 1.31.1 both affected).
Lingua principale
Python
Stelle
2.5k
Fork
340
Merge medio
2g 11h
PR unite (30g)
92

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di roboflow/inference

Tutte le issue di roboflow/inference

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.