Repository metrics
- Stars
- (1,408 stars)
- PR merge metrics
- (平均マージ 5d 5h) (30d で 16 merged PRs)
説明
Describe the bug
Accessing an index of an array in shared memory (allocated outside) a kernel throws an illegal memory access error. If a cuDeviceArray (array in GPU shared memory) is passed to a simple kernel setting (or getting) its values, this error is encountered. The error doesn't show itself till after the kernel is compiled and run and any other CUDA operation is performed (another kernel run, or just CUDA.synchronize()).
To reproduce
import CUDA
function kernel(arr)
i = CUDA.threadIdx().x
arr[i] = 1.0
return
end
arr = rand(20)
arrshared = CUDA.@cuStaticSharedMem(eltype(arr), size(arr))
copyto!(arrshared, arr)
CUDA.@cuda threads=20 kernel(arrshared)
CUDA.synchronize() # needed to make error message show
throws the error:
ERROR: CUDA error: an illegal memory access was encountered (code 700, ERROR_ILLEGAL_ADDRESS)
Stacktrace:
[1] throw_api_error(res::CUDA.cudaError_enum)
@ CUDA ~/.julia/packages/CUDA/02Kjq/lib/cudadrv/error.jl:105
[2] query
@ ~/.julia/packages/CUDA/02Kjq/lib/cudadrv/stream.jl:102 [inlined]
[3] synchronize(stream::CUDA.CuStream; blocking::Bool)
@ CUDA ~/.julia/packages/CUDA/02Kjq/lib/cudadrv/stream.jl:117
[4] synchronize (repeats 2 times)
@ ~/.julia/packages/CUDA/02Kjq/lib/cudadrv/stream.jl:117 [inlined]
[5] top-level scope
@ ~/.julia/packages/CUDA/02Kjq/src/initialization.jl:54
Manifest.toml:
[[CUDA]]
deps = ["AbstractFFTs", "Adapt", "BFloat16s", "CEnum", "CompilerSupportLibraries_jll", "DataStructures", "ExprTools", "GPUArrays", "GPUCompiler", "LLVM", "LazyArtifacts", "Libdl", "LinearAlgebra", "Logging", "Printf", "Random", "Random123", "RandomNumbers", "Reexport", "Requires", "SparseArrays", "SpecialFunctions", "TimerOutputs"]
git-tree-sha1 = "8ef71bf6d6602cf227196b43650924bf9ef7babc"
uuid = "052768ef-5323-5732-b1bb-66c8b64840ba"
version = "3.3.3"
Device: NVIDIA GTX 1080 OS: Ubuntu 18.04 LTS
Expected behavior
The shared memory can be allocated outside the kernel and used inside it to keep the data in shared rather than global memory. No error is thrown, the kernel successfully sets the value of each index of the shared array.
Version info
Details on Julia:
julia> versioninfo()
Julia Version 1.6.1
Commit 6aaedecc44 (2021-04-23 05:59 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Details on CUDA:
julia> CUDA.versioninfo()
CUDA toolkit 11.3.1, artifact installation
CUDA driver 11.2.0
NVIDIA driver 460.80.0
Libraries:
- CUBLAS: 11.5.1
- CURAND: 10.2.4
- CUFFT: 10.4.2
- CUSOLVER: 11.1.2
- CUSPARSE: 11.6.0
- CUPTI: 14.0.0
- NVML: 11.0.0+460.80
- CUDNN: 8.20.0 (for CUDA 11.3.0)
- CUTENSOR: 1.3.0 (for CUDA 11.2.0)
Toolchain:
- Julia: 1.6.1
- LLVM: 11.0.1
- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3, 6.4, 6.5, 7.0
- Device capability support: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80
1 device:
0: GeForce GTX 1080 (sm_61, 7.442 GiB / 7.921 GiB available)
(@v1.6) pkg> status
Status `~/.julia/environments/v1.6/Project.toml`
[052768ef] CUDA v3.3.3
Am I using shared memory wrong? The same code works if arrshared is just a CuArray (in global memory rather than shared). Could this be a issue specifically with my device? This isn't specific to setting the value, getindex will give the same error as setindex!, here.