JuliaGPU/CUDA.jl

Don't sync during graph record

Open

#1,229 opened on 2021年11月3日

GitHub で見る
 (0 comments) (0 reactions) (0 assignees)Julia (274 forks)batch import
buggood first issue

Repository metrics

Stars
 (1,408 stars)
PR merge metrics
 (平均マージ 5d 5h) (30d で 16 merged PRs)

説明

We eagerly sync during copyto to avoid blocking, but that doesn't work when recording a graph:

julia> @captured CuArray([1])
ERROR: CUDA error: operation failed due to a previous error during capture (code 901, ERROR_STREAM_CAPTURE_INVALIDATED)
Stacktrace:
 [1] throw_api_error(res::CUDA.cudaError_enum)
   @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/error.jl:91
 [2] capture(f::var"#11#13"; flags::CUDA.CUstreamCaptureMode_enum, throw_error::Bool)
   @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/graph.jl:44
 [3] top-level scope
   @ ~/Julia/pkg/CUDA/lib/cudadrv/graph.jl:213
 [4] top-level scope
   @ ~/Julia/pkg/CUDA/src/initialization.jl:52

caused by: CUDA error: operation not permitted when stream is capturing (code 900, ERROR_STREAM_CAPTURE_UNSUPPORTED)
Stacktrace:
  [1] throw_api_error(res::CUDA.cudaError_enum)
    @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/error.jl:91
  [2] isdone
    @ ~/Julia/pkg/CUDA/lib/cudadrv/stream.jl:107 [inlined]
  [3] nonblocking_synchronize
    @ ~/Julia/pkg/CUDA/lib/cudadrv/stream.jl:137 [inlined]
  [4] macro expansion
    @ ~/Julia/pkg/CUDA/src/array.jl:388 [inlined]
  [5] macro expansion
    @ ~/Julia/pkg/CUDA/src/state.jl:182 [inlined]
  [6] unsafe_copyto!(dest::CuArray{Int64, 1, CUDA.Mem.DeviceBuffer}, doffs::Int64, src::Vector{Int64}, soffs::Int64, n::Int64)
    @ CUDA ~/Julia/pkg/CUDA/src/array.jl:384
  [7] copyto!
    @ ~/Julia/pkg/CUDA/src/array.jl:342 [inlined]

コントリビューターガイド