Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

DeepSeek-V4.1's fp4_e4m3_b16 pack rounds a tied scale up on pre-sm_89 GPUs

オープン
  • 何か問題がありますか？
#673 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
半日
初心者へのやさしさ
42/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
python, pytorch
領域
machine-learning

調査の方向性

Start with test_roundtrip_matches_the_reference_quantizer in tests/kernels/test_dsv41_pack.py, which fails for the fp4_e4m3_b16 cases at 128 and 512. Then read _pack_rows_kernel and round_e4m3 in the kernel module, where the scale is computed as amax * (1/6) and the subnormal rounding is fused into an add. Run the test with FREETOKEN_FORCE_E4M3_EMU=1 on a GPU to reproduce. Done means both cases pass and the scales match torch's cast on tied midpoints.

索引モデルが issue の本文から書いたものです。

説明

WSL
Before you start
  • I have read the FAQ and my problem is not answered there.
  • I have read the Roadmap and this is not already planned there.
  • I have searched existing issues and found no duplicate.
  • I am on the latest release, or on a freshly rebuilt main when building from source.
What happened

On an RTX 3090 Ti (sm_86), test_roundtrip_matches_the_reference_quantizer[128-fp4_e4m3_b16] and [512-fp4_e4m3_b16] in tests/kernels/test_dsv41_pack.py fail with fp4_e4m3_b16: 0.015625. The kernel stores some group scales one e4m3 step above the reference's: 2 of 2,400 groups at dim 128 and 16 of 9,600 at dim 512. In every one, amax / 6 lies exactly on an e4m3 midpoint. For example, amax 0.052734375 gives 0.0087890625, halfway between the subnormals 0.0078125 and 0.009765625. .to(torch.float8_e4m3fn) rounds to the even 0.0078125, but the kernel stores 0.009765625.

The emulated path does this. _pack_rows_kernel rounds the scale with _e4m3_round(amax * (1.0 / 6.0)), and round_e4m3's subnormal branch is (x + 24576.0) - 24576.0. Triton fuses the multiply into that add: a kernel computing round_e4m3(amax * (1.0 / 6.0)) compiles the add to fma.rn.f32 %r8, %r1, 0f3E2AAAAB, 0f46C00000. The add therefore rounds the unrounded product, which lies just above the midpoint because 1/6 rounds up in fp32, and gets the odd code. The same value loaded from memory rounds to 0.0078125. The native branch converts in hardware and does not take this path.

I tested three fixes here; each rounds both tied values I hit (amax 0.052734375 and 0.076171875) as torch's cast does:

  1. Divide as the reference does: s = _e4m3_round(tl.math.div_rn(amax, 6.0)), as the kernel already does for the codes. With this change the test file passes, 10 of 10.
  2. Make round_e4m3's subnormal branch unfusable with libdevice.sub_rn(libdevice.add_rn(x, 24576.0), 24576.0). That keeps its round-to-nearest-even result whatever the caller computes its argument with. The other callers clamp before rounding, so no multiply reaches the add there today.
  3. Compile with enable_fp_fusion=False. Triton 3.9 turns fusion off by default (#629), which would hide this.
How did you install FreeToken

Built from source

FreeToken version

9c635a1

OS

WSL2 on Windows 11

OS details

Ubuntu 24.04, Python 3.12, torch 2.14.0+cu130, triton 3.8.0. Native Windows 11 (Python 3.13, triton-windows 3.8.0, with the Windows PRs from #384) gives the same scales.

GPU and driver

RTX 3090 Ti 24 GB, driver 617.14

CPU and system RAM

AMD Ryzen 9 9950X3D, 256 GB

Checkpoint

None; the kernel test runs without one.

Command
pytest -q --tb=line -rf tests/kernels/test_dsv41_pack.py
Full log
..F...F...                                                               [100%]
=================================== FAILURES ===================================
/home/yevhenii/FreeToken/tests/kernels/test_dsv41_pack.py:73: AssertionError: fp4_e4m3_b16: 0.015625
/home/yevhenii/FreeToken/tests/kernels/test_dsv41_pack.py:73: AssertionError: fp4_e4m3_b16: 0.015625
=========================== short test summary info ============================
FAILED tests/kernels/test_dsv41_pack.py::test_roundtrip_matches_the_reference_quantizer[128-fp4_e4m3_b16]
FAILED tests/kernels/test_dsv41_pack.py::test_roundtrip_matches_the_reference_quantizer[512-fp4_e4m3_b16]
2 failed, 8 passed in 3.93s
Anything else

FREETOKEN_FORCE_E4M3_EMU=1 selects the same emulated path, so it should reproduce on a newer GPU too; I could only run it on sm_86.

主要言語
Python
スター
14.2k
フォーク
1.4k
平均マージ
21時間 39分
マージ済み PR（30日）
100

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

FlashML-org/FreeToken のほかの issue

FlashML-org/FreeToken の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。