First kernel launch recompiles ~2.7 s of precompiled compiler code on Julia 1.12

Aperta
  • Qualcosa non va?
#593 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
julia
Ambito
backend, performance

Direzione di ricerca

Start by reproducing the first-launch compile timing with the Julia snippet on 1.12.7 and 1.13.2, then inspect the listed invalidation sources: GPUArrays src/host/mapreduce.jl, LLVM.jl src/core/type.jl, and OpenCL.jl src/compiler/precompile.jl. The issue identifies fixes across GPUArrays, Base.Compiler, LLVM.jl, and OpenCL.jl, so first trace which invalidations still occur. Done means addressing the invalidations and adding a fresh-session test that verifies the first kernel launch does not recompile.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

On Julia 1.12, the first kernel launch of a fresh session spends 2.6–2.7 s recompiling code that was already precompiled. Most of that is GPUCompiler's inference code, which gets invalidated when GPUArrays is loaded.

using OpenCL, pocl_jll
a = CLArray([1f0])
Base.cumulative_compile_timing(true)
c0, r0 = Base.cumulative_compile_time_ns()
t = @elapsed (b = a .+ 1f0; Array(b))
c1, r1 = Base.cumulative_compile_time_ns()
println(VERSION, ": first broadcast ", round(Int, t*1000), " ms, compile ", round(Int, (c1-c0)/1e6), " ms, recompile ", round(Int, (r1-r0)/1e6), " ms")

OpenCL.jl 1.0.0, pocl_jll 7.2.2, GPUCompiler 2.18.0, GPUArrays 12.1.1, KernelAbstractions 0.10.3, on an M3 Pro (PoCL CPU device):

1.12.7: first broadcast 5668 ms, compile 5586 ms, recompile 2630 ms
1.13.2: first broadcast 3794 ms, compile 3755 ms, recompile 221 ms

--trace-compile-timing on 1.12.7 shows these entries marked # recompile:

2202.2 ms  Base.Compiler.typeinf_local(::GPUCompiler.GPUInterpreter{CachedMethodTable{StackedMethodTable{…}}}, …)
 135.2 ms  GPUCompiler.AddKernelState(::LLVM.Module)
  22.3 ms  GPUCompiler.LowerKernelState(::LLVM.Function)
   7.6 ms  GPUCompiler.LinkRuntime(::LLVM.Module)
   6.7 ms  Base.Compiler.finish_nocycle(::GPUInterpreter{…}, …)
   3.6 ms  kwcall(typeinf_type; interp::GPUInterpreter{…})

What gets invalidated, from SnoopCompile on using OpenCL, pocl_jll:

  • typeinf_local and friends (1.12 only). KernelAbstractions' precompile workload compiles a kernel for its POCL back-end, which puts typeinf_local(::GPUInterpreter{…}) into KA's package image. The interpreter type is the same one OpenCL.jl uses. GPUArrays, which loads after KA, adds any(f::Function, A::AnyGPUArray; dims) (src/host/mapreduce.jl:180). That method supersedes the any(::closure, ::AbstractArray) instances that Base.Compiler's inference calls (compileable_specialization and others), 1627 invalidations in total. It is the only inserted method whose invalidation tree reaches typeinf_local(::GPUInterpreter…). On 1.13 it no longer reaches it. The fix probably belongs in GPUArrays (a narrower any signature) or in Base.Compiler (a concretely typed any), not in OpenCL.jl.
  • AddKernelState and LowerKernelState, on 1.12 and 1.13. They are invalidated by the kernel_state_type(::CompilerJob{SPIRVCompilerTarget, …}) methods that KernelAbstractions' POCL back-end and OpenCL.jl add.
  • compile_or_lookup(::CompilerJob{SPIRVCompilerTarget, OpenCLCompilerParams}), which src/compiler/precompile.jl precompiles, is also invalidated, when AcceleratedKernels' OpenCLExt adds reduce_tuning(::OpenCLBackend, ::Type). The abstract call that leads there is in LLVM.jl: isemptytype(::StructType) = all(isemptytype, elements(T)) (src/core/type.jl:630) recurses into itself, so inference widens it to all(::Function, ::StructTypeElementSet), which reaches GPUArrays' and AcceleratedKernels' reductions. CUDA.jl, oneAPI.jl and Metal.jl are affected by the same thing on 1.12. Rewriting it as a loop fixes it there; for CUDA.jl that removes ~300 ms of recompilation from the first launch.

Separately, OpenCL.jl's precompile workload doesn't compile or launch a kernel, so the first launch always compiles clfunction's lookup and link path from scratch. Before adding that to the workload, the invalidations above need to be fixed, or that code would be recompiled anyway. Metal.jl went through this in JuliaGPU/Metal.jl#1116, which also adds a test that the first launch in a fresh session doesn't recompile anything.

Lingua principale
Julia
Stelle
313
Fork
50
Merge medio
1g 8h
PR unite (30g)
44

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di JuliaGPU/OpenCL.jl

Tutte le issue di JuliaGPU/OpenCL.jl

Issue simili

Altre issue su Julia

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.