Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

CPU backend: kernels are vectorized with 256-bit vectors on AVX-512 CPUs where Julia uses 512-bit ones

Open
  • Is something wrong?
#891 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
28/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Domain
compilers, performance

Research direction

Start by running the MWE with POCL_LEAVE_KERNEL_COMPILER_TEMP_FILES=1 and a separate POCL_CACHE_DIR, then inspect the cached kernel object for ymm versus zmm use. The payload suggests the lever is the CPU name PoCL reports (cpu-skylake-avx512 on a Zen 5 host), not the forced vector width, which had no effect. Done means the kernel's timing matches plain Julia on the reported CPU, which requires understanding PoCL's CPU target selection.

Written by the indexing model from the issue text.

Description

kernels performance pocl regression upstream

On a Zen 5 CPU, PoCL reports the device as cpu-skylake-avx512-AMD Ryzen Threadripper PRO 9955WX 16-Cores and vectorizes kernels with ymm registers, presumably because LLVM prefers 256-bit vectors for skylake-avx512. Julia targets znver5 and uses zmm. A compute-bound loop in a kernel is then 1.3× slower than the same loop in plain Julia, and than the same kernel with 0.9.45. Forcing the vector width closes the gap.

MWE
# s[:, i] = k * q[:, i]: a compute-bound loop nest inside each work-item.
using KernelAbstractions, BenchmarkTools, Printf

@kernel inbounds=true function matvec_cols!(s, @Const(k), @Const(q))
    i = @index(Global, Linear)
    for d in 1:size(k, 2)
        qd = q[d, i]
        @simd for j in 1:size(k, 1)
            s[j, i] = muladd(qd, k[j, d], s[j, i])
        end
    end
end

function matvec_julia!(s, k, q)
    @inbounds for i in axes(q, 2), d in axes(k, 2)
        qd = q[d, i]
        @simd for j in axes(k, 1)
            s[j, i] = muladd(qd, k[j, d], s[j, i])
        end
    end
end

n, D, m = 128, 64, 256
k = rand(Float32, n, D); q = rand(Float32, D, m); s = zeros(Float32, n, m)
backend = CPU()
kernel! = matvec_cols!(backend)
tj = @belapsed matvec_julia!($s, $k, $q)
tk = @belapsed ($kernel!($s, $k, $q; ndrange=$m); KernelAbstractions.synchronize($backend))
println("KernelAbstractions ", pkgversion(KernelAbstractions), ", Julia ", VERSION, ", ", Threads.nthreads(), " thread, ", Sys.CPU_NAME)
@printf("plain Julia loop %6.1f µs   kernel %6.1f µs   (%.2fx)\n", 1e6 * tj, 1e6 * tk, tk / tj)

Julia 1.13.1, 1 thread, pocl_standalone_jll 7.2.2+7, Linux:

time multiply-add in the loop
plain Julia loop 62.6 µs zmm
kernel, 0.9.45 63.3 µs
kernel, 0.10.0 82.7 µs ymm
kernel, 0.10.0, POCL_DEBUG_LLVM_OPTS=force-vector-width=16 54.4 µs zmm

The registers of the kernels are those in the object file PoCL leaves in its cache (POCL_LEAVE_KERNEL_COMPILER_TEMP_FILES=1, a separate POCL_CACHE_DIR per setting). POCL_DEBUG_LLVM_OPTS=prefer-vector-width=512 has no effect.

The distro build of PoCL seems to map every AVX-512 host to skylake-avx512. Tuning for the host CPU, or taking the preferred vector width from it, would avoid this on Zen 4 and Zen 5, where 512-bit vectors do not throttle.

Related: #865, #860. Found while measuring the attention kernels of FluxML/NNlib.jl#791 on the CPU backend.

Dominant language
Julia
Stars
532
Forks
88
Avg merge
1d 12h
Merged PRs (30d)
72

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from JuliaGPU/KernelAbstractions.jl

All issues in JuliaGPU/KernelAbstractions.jl

Similar issues

More Julia issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.