flashinfer-ai/flashinfer

flashinfer.nvfp4_quantize: make CuTe-DSL backend's `a_global_sf` arg a host side arg (to align CuTe-DSL backend with CUDA backend)

已关闭

#4,112 创建于 2026年7月23日

 (2 条评论) (0 个反应) (1 位负责人)Python (1,031 个派生)github user discovery
good first issueop: misc

仓库指标

星标
 (5,756 个星标)
PR 合并指标
 (平均合并 11天 23小时) (30 天内合并 186 个 PR)

描述

As reported: "In flashinfer.nvfp4_quantize, the type of the argument a_global_sf is inconsistent. For cuda, it is a host side value; for cute-dsl, it is a device side tensor. The function description is not clear enough. We should make it host side and passed as a function argument.

What we might be able to do relatively easily is to change the cute-dsl backend kernel to accept either (a) a device side tensor of size 1 and dtype float32 (what we support today); or (b) a single float value that trivially comes from host.  That way it would be backwards compatible for those who might be using the device-side tensor."

贡献者指南