JuliaGPU/CUDA.jl

Constructing shared memory on the CPU should fail

Open

#1,047 opened on Jul 11, 2021

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Julia (274 forks)batch import
enhancementgood first issue

Repository metrics

Stars
 (1,408 stars)
PR merge metrics
 (Avg merge 5d 5h) (16 merged PRs in 30d)

Description

Describe the bug

Accessing an index of an array in shared memory (allocated outside) a kernel throws an illegal memory access error. If a cuDeviceArray (array in GPU shared memory) is passed to a simple kernel setting (or getting) its values, this error is encountered. The error doesn't show itself till after the kernel is compiled and run and any other CUDA operation is performed (another kernel run, or just CUDA.synchronize()).

To reproduce


import CUDA

function kernel(arr)
        i = CUDA.threadIdx().x
        arr[i] = 1.0
        return
end

arr = rand(20)

arrshared = CUDA.@cuStaticSharedMem(eltype(arr), size(arr))

copyto!(arrshared, arr)

CUDA.@cuda threads=20 kernel(arrshared)

CUDA.synchronize() # needed to make error message show

throws the error:

ERROR: CUDA error: an illegal memory access was encountered (code 700, ERROR_ILLEGAL_ADDRESS)
Stacktrace:
 [1] throw_api_error(res::CUDA.cudaError_enum)
   @ CUDA ~/.julia/packages/CUDA/02Kjq/lib/cudadrv/error.jl:105
 [2] query
   @ ~/.julia/packages/CUDA/02Kjq/lib/cudadrv/stream.jl:102 [inlined]
 [3] synchronize(stream::CUDA.CuStream; blocking::Bool)
   @ CUDA ~/.julia/packages/CUDA/02Kjq/lib/cudadrv/stream.jl:117
 [4] synchronize (repeats 2 times)
   @ ~/.julia/packages/CUDA/02Kjq/lib/cudadrv/stream.jl:117 [inlined]
 [5] top-level scope
   @ ~/.julia/packages/CUDA/02Kjq/src/initialization.jl:54

Manifest.toml:

[[CUDA]]
deps = ["AbstractFFTs", "Adapt", "BFloat16s", "CEnum", "CompilerSupportLibraries_jll", "DataStructures", "ExprTools", "GPUArrays", "GPUCompiler", "LLVM", "LazyArtifacts", "Libdl", "LinearAlgebra", "Logging", "Printf", "Random", "Random123", "RandomNumbers", "Reexport", "Requires", "SparseArrays", "SpecialFunctions", "TimerOutputs"]
git-tree-sha1 = "8ef71bf6d6602cf227196b43650924bf9ef7babc"
uuid = "052768ef-5323-5732-b1bb-66c8b64840ba"
version = "3.3.3"

Device: NVIDIA GTX 1080 OS: Ubuntu 18.04 LTS

Expected behavior

The shared memory can be allocated outside the kernel and used inside it to keep the data in shared rather than global memory. No error is thrown, the kernel successfully sets the value of each index of the shared array.

Version info

Details on Julia:

julia> versioninfo()
Julia Version 1.6.1
Commit 6aaedecc44 (2021-04-23 05:59 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)

Details on CUDA:

julia> CUDA.versioninfo()

CUDA toolkit 11.3.1, artifact installation
CUDA driver 11.2.0
NVIDIA driver 460.80.0

Libraries: 
- CUBLAS: 11.5.1
- CURAND: 10.2.4
- CUFFT: 10.4.2
- CUSOLVER: 11.1.2
- CUSPARSE: 11.6.0
- CUPTI: 14.0.0
- NVML: 11.0.0+460.80
- CUDNN: 8.20.0 (for CUDA 11.3.0)
- CUTENSOR: 1.3.0 (for CUDA 11.2.0)

Toolchain:
- Julia: 1.6.1
- LLVM: 11.0.1
- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3, 6.4, 6.5, 7.0
- Device capability support: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80

1 device:
  0: GeForce GTX 1080 (sm_61, 7.442 GiB / 7.921 GiB available)
(@v1.6) pkg> status
        Status `~/.julia/environments/v1.6/Project.toml`
     [052768ef] CUDA v3.3.3

Am I using shared memory wrong? The same code works if arrshared is just a CuArray (in global memory rather than shared). Could this be a issue specifically with my device? This isn't specific to setting the value, getindex will give the same error as setindex!, here.

Contributor guide