JuliaGPU/CUDA.jl

Don't sync during graph record

Open

#1 229 ouverte le 3 nov. 2021

Voir sur GitHub
 (0 commentaires) (0 réactions) (0 assignés)Julia (274 forks)batch import
buggood first issue

Métriques du dépôt

Stars
 (1 408 stars)
Métriques de merge PR
 (Merge moyen 5j 5h) (16 PRs mergées en 30 j)

Description

We eagerly sync during copyto to avoid blocking, but that doesn't work when recording a graph:

julia> @captured CuArray([1])
ERROR: CUDA error: operation failed due to a previous error during capture (code 901, ERROR_STREAM_CAPTURE_INVALIDATED)
Stacktrace:
 [1] throw_api_error(res::CUDA.cudaError_enum)
   @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/error.jl:91
 [2] capture(f::var"#11#13"; flags::CUDA.CUstreamCaptureMode_enum, throw_error::Bool)
   @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/graph.jl:44
 [3] top-level scope
   @ ~/Julia/pkg/CUDA/lib/cudadrv/graph.jl:213
 [4] top-level scope
   @ ~/Julia/pkg/CUDA/src/initialization.jl:52

caused by: CUDA error: operation not permitted when stream is capturing (code 900, ERROR_STREAM_CAPTURE_UNSUPPORTED)
Stacktrace:
  [1] throw_api_error(res::CUDA.cudaError_enum)
    @ CUDA ~/Julia/pkg/CUDA/lib/cudadrv/error.jl:91
  [2] isdone
    @ ~/Julia/pkg/CUDA/lib/cudadrv/stream.jl:107 [inlined]
  [3] nonblocking_synchronize
    @ ~/Julia/pkg/CUDA/lib/cudadrv/stream.jl:137 [inlined]
  [4] macro expansion
    @ ~/Julia/pkg/CUDA/src/array.jl:388 [inlined]
  [5] macro expansion
    @ ~/Julia/pkg/CUDA/src/state.jl:182 [inlined]
  [6] unsafe_copyto!(dest::CuArray{Int64, 1, CUDA.Mem.DeviceBuffer}, doffs::Int64, src::Vector{Int64}, soffs::Int64, n::Int64)
    @ CUDA ~/Julia/pkg/CUDA/src/array.jl:384
  [7] copyto!
    @ ~/Julia/pkg/CUDA/src/array.jl:342 [inlined]

Guide contributeur