Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Launching a kernel with a new signature compiles a lot of signature-independent host code

Open
  • Is something wrong?
#1,153 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Active
Tech stack
julia
Domain
performance

Research direction

Start with hipfunction and _kernel_instances in src/compiler/codegen.jl, then compare the analogous cufunction and cufunction_link patterns in CUDA.jl. Read launch_tuple and pack_arguments in src/runtime/hip-execution.jl and the argument conversion in src/highlevel.jl; validate changes by measuring host-side precompile time for new kernel signatures, including through KernelAbstractions. Done means reducing compilation overhead while accounting for the small per-launch allocation tradeoff.

Written by the indexing model from the issue text.

Description

JuliaGPU/Metal.jl#1014 cut the host-side compilation needed to launch a kernel with a new argument signature by 4-10x on Metal. I checked which of the causes found there also apply to AMDGPU.jl; two do.

These numbers come from a machine without an AMD GPU: they time precompile of a function doing @roc groupsize=64 kern(y, x, a) for a new kernel signature, i.e., host-side inference and code generation only, with the changes below prototyped by redefining the functions. A new signature currently costs ~95 ms (Julia 1.12) / ~101 ms (1.13), and ~79 / ~86 ms through KernelAbstractions.

  • hipfunction (src/compiler/codegen.jl:275-290) compiles its whole body for every kernel: the lock, device and config lookup, hipfunction_lookup, and the _kernel_instances::Dict{Any,Any} cache (codegen.jl:48), which is keyed by a tuple containing f, so hashing and the Dict accesses are compiled for every kernel too. A HIPKernel is only (f, fun), so that cache can go. Splitting it into a thin hipfunction that only does methodinstance(F, tt) and constructs the HIPKernel, with the rest in an unspecialized @noinline function (like cufunction and cufunction_link in CUDA.jl), brings a new signature from 95 to 61 ms.
  • launch_tuple(fun::HIPFunction, args::Tuple; ...) (src/runtime/hip-execution.jl:125-154) is specialized on the converted arguments, but only pack_arguments needs them (~25 ms exclusive in the inference profile). Packing in a specialized function and launching through a @noinline function taking the parameter pointer brings it from 95 to 68 ms. This gives up the alloca mentioned at hip-execution.jl:138-140, i.e., costs a small allocation per launch.

With both changes, a new signature costs 37 / 42 ms, and 40 / 50 ms through KernelAbstractions.

Unrelated, but noticed along the way: @roc converts the arguments once to compute their types (src/highlevel.jl:155) and again when calling the kernel (hip-execution.jl:70).

Dominant language
Julia
Stars
346
Forks
81
Avg merge
2d 5h
Merged PRs (30d)
52

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from JuliaGPU/AMDGPU.jl

All issues in JuliaGPU/AMDGPU.jl

Similar issues

More Julia issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.