JuliaGPU/CUDA.jl

Graph memory management

Open

#1,230 建立於 2021年11月3日

在 GitHub 查看
 (0 留言) (0 反應) (0 負責人)Julia (274 fork)batch import
buggood first issue

倉庫指標

Star
 (1,408 star)
PR 合併指標
 (平均合併 5天 5小時) (30 天內合併 16 個 PR)

描述

There's something wrong with how we do allocations during graph capture; I'm noticing OOMs as a result. I think the issue is with allocating during record:

julia> f() = @captured CuArray{Int}(undef, 1024, 1024)
f (generic function with 1 method)

julia> f()

According to the docs:

During stream capture, [cuMemAllocAsyncP results in the creation of an allocation node. In this case, the allocation is owned by the graph instead of the memory pool. The memory pool's properties are used to set the node's creation parameters.

We actually do free the initial allocation that occurred during recording, but that seems to throw an error:

julia> GC.gc()
WARNING: Error while freeing DeviceBuffer(8 bytes at 0x0000000ae2000000):
CUDA.CuError(code=CUDA.cudaError_enum(0x00000001), meta=nothing)

0x1 is INVALID_VALUE

Worse, subsequent launches of the graph result in new allocations which go untracked!

julia> while true
       f()
       end
ERROR: Out of GPU memory

Again, quoting the docs:

Allocations which are not freed in the same graph can be freed by:

  • passing the allocation to cuMemFreeAsync or cuMemFree;
  • launching a graph with a free node for that allocation; or
  • specifying CUDA_GRAPH_INSTANTIATE_FLAG_AUTO_FREE_ON_LAUNCH during instantiation, which makes each launch behave as though it called cuMemFreeAsync for every unfreed allocation.

貢獻者指南