onnxruntime CoreML execution provider leaks ~70-78 KB per inference on macOS (RF-DETR default config)

Open
  • Is something wrong?
#3,302 13 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Nishant-ZFYII is already working on this.

Since Oct 30, 2026.

  • #3402 by @davidnichols-ops — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
macos, python
Domain
performance

Research direction

Start by reading inference_models/inference_models/models/common/onnx.py, especially get_default_coreml_provider_options, to understand the CoreML options used by default. The standalone reproduction script is on the reporter’s machine and can be attached on request; first ask for it, then compare memory growth with CoreML and CPU providers. Done means the native memory growth per inference is explained and a fix or agreed mitigation is established.

Written by the indexing model from the issue text.

Description

Summary

On macOS, running an ONNX model through onnxruntime's CoreMLExecutionProvider with the options inference_models sets by default for RF-DETR (ModelFormat=MLProgram, MLComputeUnits=CPUAndGPU, see inference_models/inference_models/models/common/onnx.py, get_default_coreml_provider_options) grows process memory by roughly 70-78 KB per session.run call. At 30 inferences per second that is about 1.9-2.0 MB/s, or 6-7 GB per day. The leak is native (not in the Python heap) and is present without any Roboflow code in the loop.

Found while validating an unrelated Workflows feature; it is not caused by that work.

Evidence

Mac M4 Max, macOS 27.0.1, Python 3.10, rfdetr-nano (coco/40 weights). Memory is phys_footprint sampled every second; slopes are linear fits for t >= 30 s.

Run Growth
InferencePipeline + RF-DETR + visualizations, pristine main (18eb2936e) +1.95 MB/s, R² 0.98
Model only through the workflow engine +1.90 MB/s, R² 0.99
Model only with ONNXRUNTIME_EXECUTION_PROVIDERS=CPUExecutionProvider flat (-0.18 MB/s)
Passthrough workflow, no model flat
Bare onnxruntime 1.21.1 loop, CoreML MLProgram CPUAndGPU, no Roboflow code +1.43 MB/s at 18.4 it/s = 78 KB per call, R² 1.00
Same loop, onnxruntime 1.31.1 (latest) +1.96 to 2.15 MB/s at 28.7 it/s
CoreML MLProgram, MLComputeUnits=CPUOnly flat, 14.8 it/s
CoreML NeuralNetwork format flat, but 3 it/s and a 3.7 GB footprint
Objective-C autorelease pool pushed/popped around each run still +2.35 MB/s

tracemalloc shows the Python heap shrinking while RSS grows (94.8 to 84.3 MB traced vs +180 MB process over the same 180 s), so nothing Python-side accumulates. The leak is in the MLProgram GPU path.

Repro

A standalone script that loads the cached rfdetr-nano weights with the same CoreML options and runs a paced loop while sampling memory lives at ~/Code/vision-event-clips/scripts/g1.9/ort_loop.py on Brad's machine (can be attached on request):

python ort_loop.py --provider coreml --seconds 300 --fps 30 --label coreml   # leaks
python ort_loop.py --provider cpu    --seconds 300 --fps 30 --label cpu      # flat

Impact

Any long-running RF-DETR (or other MLProgram-opted-in model) deployment on macOS with the default provider options, e.g. InferencePipeline on a Mac for hours.

Mitigation candidates

  • MLComputeUnits=CPUOnly for CoreML: no leak, about half the throughput (14.8 vs 30 it/s on this model).
  • Periodic InferenceSession recycling in the ONNX model wrapper.
  • Upstream report to onnxruntime with the bare-loop repro (1.21.1 and 1.31.1 both affected).
Dominant language
Python
Stars
2.5k
Forks
340
Avg merge
2d 11h
Merged PRs (30d)
92

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from roboflow/inference

All issues in roboflow/inference

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.