`copy` and `resize!` of a `ROCArray{T,N,Mem.HostBuffer}` silently switch to `Mem.HIPBuffer` / throw a convert MethodError
Maintainers usually reply within 1 day
Assessment
This issue has not been assessed yet.
Description
ROCArray{T,N,B} carries its buffer type B, and similar preserves it, but copy and resize! do not:
using AMDGPU
x = ROCArray{Float32,1,AMDGPU.Mem.HostBuffer}(undef, 4); fill!(x, 1f0)
typeof(x) # ROCArray{Float32, 1, AMDGPU.Runtime.Mem.HostBuffer}
typeof(copy(x)) # ROCArray{Float32, 1, AMDGPU.Runtime.Mem.HIPBuffer} <- buffer type changed
typeof(similar(x)) # ROCArray{Float32, 1, AMDGPU.Runtime.Mem.HostBuffer} (preserved)
resize!(x, 8)
# ERROR: MethodError: Cannot `convert` an object of type
# GPUArrays.DataRef{AMDGPU.Managed{AMDGPU.Runtime.Mem.HIPBuffer}} to an object of type
# GPUArrays.DataRef{AMDGPU.Managed{AMDGPU.Runtime.Mem.HostBuffer}}
# Stacktrace: setproperty!(::ROCArray{…,HostBuffer}, :data, ::DataRef{Managed{HIPBuffer}}) in resize!
y = ROCArray{Float32,1,AMDGPU.Mem.HIPBuffer}(undef, 4); resize!(y, 8) # fine
Cause (as of main 284e548): Base.copy(X::ROCArray{T}) allocates ROCArray{T}(undef, size(X)), i.e. the default
HIPBuffer (src/array.jl:278-282); Base.resize!(A::ROCVector{T}, n) allocates
new_buf = Mem.HIPBuffer(bufsize; stream=stream()) unconditionally and assigns the new DataRef to a field typed
with the original buffer (src/array.jl:555-566). The TODO comment above resize! already says "Specialize ROCArray
on storage type / Check that it is not HostBuffer".
Consequence downstream: a struct holding HostBuffer arrays cannot be deepcopy'd (typeassert in
deepcopy_internal fails on the changed type) and its vectors cannot be grown with resize!, so host-coherent
arrays are not a drop-in for code written against ROCArray (seen with a Pauli-propagation vector backend on an
MI300A APU, where host-coherent memory is the natural choice). Related: #840 (unified memory support).
Expected: copy returns the same buffer type (as similar does); resize! either keeps the buffer type or throws
an ArgumentError stating that HostBuffer arrays cannot be resized.
Environment: AMDGPU.jl v2.8.0 (LLVM.jl 9.13.2; the same code is on main 284e548), Julia 1.11.6 (official x86_64
tarball), ROCm 7.2.4, HIP 7.2.53211, rocBLAS 5.2.0, AMD Instinct MI300A (gfx942:sramecc+:xnack-, 4 devices,
128 GiB each), AlmaLinux 9.8, kernel 5.14. AMDGPU.functional() == true. Note: Julia 1.11's LLVM prints
"'gfx942' is not a recognized processor for this target" during compilation; unrelated to this issue (kernels run).
- Dominant language
- Julia
- Stars
- 348
- Forks
- 81
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 54
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from JuliaGPU/AMDGPU.jl
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
JuliaGPU/AMDGPU.jl#1040 · 8 comments ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 54/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 1 day
All issues in JuliaGPU/AMDGPU.jl
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
OpenSourceAWE/SymbolicAWEModels.jl#431 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
EnzymeAD/Reactant.jl#3549 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
WaterLily-jl/WaterLily-Examples#46 · 1 comment ·