flashinfer-ai/flashinfer

[Compilation] GDN decode cold kernel compilation time is too long on SM90/100/120/121

オープン

#4,110 opened on 2026/07/23

 (12 件のコメント) (0 件のリアクション) (1 人の担当者)Python (1,031 件のフォーク)github user discovery
good first issueop: gdn

Repository metrics

Stars
 (5,756 個のスター)
PR merge metrics
 (平均マージ 11d 23h) (30d で 186 merged PRs)

説明

Noticed by @bkryu : "In our recent CI telemetry, we have been seeing that tests/gdn/test_decode_delta_rule.py runs very long (30-50 mins on every arch) and has been straining our CI.

TL;DR of the cause: the reference and kernels are all fine. Most time is spent on cold kernel compilation. On a B200 I locally see ~800 cases that collapses to ~200 cute.compile calls based on the cache key logic.

For some kernels, B/tile_v/T/cache flags/pool-strides each key a new compile. Would be great if we can make the kernels batch and/or seqlen dynamic to systematically reduce unit testing time."

コントリビューターガイド