JuliaGPU/CUDA.jl

Add stream pool to avoid cuStreamCreate overhead

Open

#2,012 opened on 2023年7月29日

GitHub で見る
 (0 comments) (0 reactions) (0 assignees)Julia (274 forks)batch import
enhancementgood first issueperformance

Repository metrics

Stars
 (1,408 stars)
PR merge metrics
 (平均マージ 5d 5h) (30d で 16 merged PRs)

説明

Describe the bug

I tried adding multithreading to our code, and now we see occasional pauses due to CuStreamCreate:

Could be fixed by adding a Stream pool?

To reproduce

Available on request

Expected behavior

No pauses

Version info

Not sure exactly, I think it was Julia 1.8.5 with CUDA.jl 3.13.1 and CUDA 11.2.

Once our cluster upgrades the drivers, I can rerun with CUDA 12.

Additional context

cc @lcw

コントリビューターガイド