JuliaGPU/CUDA.jl

Don't sync during graph record

Open

#1229 aperta il 3 nov 2021

Vedi su GitHub
 (0 commenti) (0 reazioni) (0 assegnatari)Julia (274 fork)batch import
buggood first issue

Metriche repository

Star
 (1408 star)
Metriche merge PR
 (Merge medio 5g 5h) (16 PR mergiate in 30 g)

Descrizione

We eagerly sync during copyto to avoid blocking, but that doesn't work when recording a graph:

julia> @captured CuArray([1])
ERROR: CUDA error: operation failed due to a previous error during capture (code 901, ERROR_STREAM_CAPTURE_INVALIDATED)
Stacktrace:
 [1] throw_api_error(res::CUDA.cudaError_enum)
   @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/error.jl:91
 [2] capture(f::var"#11#13"; flags::CUDA.CUstreamCaptureMode_enum, throw_error::Bool)
   @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/graph.jl:44
 [3] top-level scope
   @ ~/Julia/pkg/CUDA/lib/cudadrv/graph.jl:213
 [4] top-level scope
   @ ~/Julia/pkg/CUDA/src/initialization.jl:52

caused by: CUDA error: operation not permitted when stream is capturing (code 900, ERROR_STREAM_CAPTURE_UNSUPPORTED)
Stacktrace:
  [1] throw_api_error(res::CUDA.cudaError_enum)
    @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/error.jl:91
  [2] isdone
    @ ~/Julia/pkg/CUDA/lib/cudadrv/stream.jl:107 [inlined]
  [3] nonblocking_synchronize
    @ ~/Julia/pkg/CUDA/lib/cudadrv/stream.jl:137 [inlined]
  [4] macro expansion
    @ ~/Julia/pkg/CUDA/src/array.jl:388 [inlined]
  [5] macro expansion
    @ ~/Julia/pkg/CUDA/src/state.jl:182 [inlined]
  [6] unsafe_copyto!(dest::CuArray{Int64, 1, CUDA.Mem.DeviceBuffer}, doffs::Int64, src::Vector{Int64}, soffs::Int64, n::Int64)
    @ CUDA ~/Julia/pkg/CUDA/src/array.jl:384
  [7] copyto!
    @ ~/Julia/pkg/CUDA/src/array.jl:342 [inlined]

Guida contributor