JuliaGPU/CUDA.jl

Graph memory management

Open

#1,230 opened on 2021年11月3日

GitHub で見る
 (0 comments) (0 reactions) (0 assignees)Julia (274 forks)batch import
buggood first issue

Repository metrics

Stars
 (1,408 stars)
PR merge metrics
 (平均マージ 5d 5h) (30d で 16 merged PRs)

説明

There's something wrong with how we do allocations during graph capture; I'm noticing OOMs as a result. I think the issue is with allocating during record:

julia> f() = @captured CuArray{Int}(undef, 1024, 1024)
f (generic function with 1 method)

julia> f()

According to the docs:

During stream capture, [cuMemAllocAsyncP results in the creation of an allocation node. In this case, the allocation is owned by the graph instead of the memory pool. The memory pool's properties are used to set the node's creation parameters.

We actually do free the initial allocation that occurred during recording, but that seems to throw an error:

julia> GC.gc()
WARNING: Error while freeing DeviceBuffer(8 bytes at 0x0000000ae2000000):
CUDA.CuError(code=CUDA.cudaError_enum(0x00000001), meta=nothing)

0x1 is INVALID_VALUE

Worse, subsequent launches of the graph result in new allocations which go untracked!

julia> while true
       f()
       end
ERROR: Out of GPU memory

Again, quoting the docs:

Allocations which are not freed in the same graph can be freed by:

  • passing the allocation to cuMemFreeAsync or cuMemFree;
  • launching a graph with a free node for that allocation; or
  • specifying CUDA_GRAPH_INSTANTIATE_FLAG_AUTO_FREE_ON_LAUNCH during instantiation, which makes each launch behave as though it called cuMemFreeAsync for every unfreed allocation.

コントリビューターガイド