Kernel launches are compiled for Vararg{Any} and dispatched at run time

已关闭
  • 哪里不对？
#675 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

已经有一个关联 PR 被合并了。

  • #687 来自 @maleadt —— 已合并

评估

难度
4/5
预计耗时
3-5 天
新手友好度
30/100
Issue 类型
重构
描述清晰度
基本清楚
活跃度
活跃
领域
compilers, performance

调研方向

Start with code_typed on a kernel call to confirm the Vararg{Any} specialization in src/oneAPIKernels.jl (around line 119) and src/compiler/execution.jl (around line 403). Then look at zefunction (lines 287-323), device() in src/context.jl, and setindex! in lib/level-zero/module.jl. Done means the changes infer fully and precompile timing drops for a new signature, which needs an Intel GPU to verify launches.

由索引模型根据 Issue 内容生成。

描述

JuliaGPU/Metal.jl#1014 cut the host-side compilation needed to launch a kernel with a new argument signature by 4-10x on Metal. I checked which of the causes found there also apply to oneAPI.jl; several do, and here they also make every launch go through dynamic dispatch.

I don't have an Intel GPU, so this is based on code_typed and on timing precompile of a function launching a new kernel signature, with the changes below prototyped by redefining the functions.

  • (obj::KA.Kernel{oneAPIBackend})(args...; ndrange, workgroupsize) (src/oneAPIKernels.jl:119) and (kernel::HostKernel)(args...; kwargs...) (src/compiler/execution.jl:403) are compiled for Vararg{Any}, e.g. (..., ::oneArray{Float32,...}, ::Vararg{Any}), even when called from concretely typed code: Julia doesn't specialize a method on varargs it only passes on. That includes the @oneapi macro expansion. Everything after that point is dispatched at run time (map(kernel_convert, ...), zefunction, call, launch_configuration), so each new signature gets compiled piecemeal during its first launch, and every launch pays for the dispatches. Metal.jl fixed this with args::Vararg{Any,N} ... where N.
  • zefunction (src/compiler/execution.jl:287-323) compiles its whole body for every kernel. device() (src/context.jl:74-78) infers as Any, which makes compiler_config, the comparisons, hash and the conversion dynamic. The get!(_kernel_instances, h) do HostKernel{F,tt}(f, fun) end closure has a new type for every kernel, so get! is compiled per kernel as well, even though a HostKernel is only (f, fun) and needs no cache. Splitting it like cufunction and cufunction_link in CUDA.jl, with device()::ZeDevice, leaves no dynamic calls on the @oneapi path.
  • setindex!(::ZeKernelArgumentList, value, i) (lib/level-zero/module.jl:208-211) is compiled per argument type with the @checked zeKernelSetArgumentValue wrapper inlined. Computing the size and address per type and calling an unspecialized @noinline setter saves ~13 ms per new signature.
  • launch_configuration (src/compiler/execution.jl:246) returns Union{Missing,Int}, because checking kernel_props.maxGroupSize !== missing doesn't narrow a later field access. As a result threads_to_workgroupsize returns Tuple{Any}, and the final kernel(ctx, args...; items, groups) in the KernelAbstractions launch is a dynamic call. Binding the field to a local first fixes that.

With the first three changes the whole path is inferred, and precompile puts a new signature at ~50 ms (Julia 1.12) / ~44 ms (1.13) for @oneapi and ~31 ms through KernelAbstractions. I can't give a meaningful number for the current code without a GPU, because most of its launch path is only compiled during the launch itself.

主要语言
Julia
星标
218
派生
36
平均合并
1 天 14 小时
30 天内合并 PR
59

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

JuliaGPU/oneAPI.jl 的其他 Issue

查看 JuliaGPU/oneAPI.jl 的全部 Issue

相似的 Issue

更多 Julia Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。