Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Finite FP16 MX blocks become NaNs on the default PyTorch path

オープン
  • 何か問題がありますか？
#46 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

@vera-rubin がすでに取り組んでいます。

2026年10月13日 から。

  • #47 @vera-rubin による — オープン

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
48/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
python, pytorch
領域
machine-learning

調査の方向性

Start with mx/mx_ops.py at the reported scale calculation around line 285, then inspect the zero guard around line 78. Run the provided reproduction on CPU and compare the six formats and the linear call with the expected finite outputs. Done means zero blocks stay zero and the small E5M2 block and public linear call return finite expected results; the report says no correction was tested.

索引モデルが issue の本文から書いたものです。

説明

Finite FP16 blocks become NaNs during MX quantization with custom_cuda=False. This includes zero blocks and a public linear call that should return zero.

Tested at microsoft/microxcaling commit 7bc41952de394f5cc5e782baf132e7c7542eb4e4 (mx==1.1.0). Both fresh environments used Python 3.12.3 and NumPy 1.26.4 on x86_64 Linux. The failure reproduced with CPU PyTorch 2.2.0+cpu and 2.4.0+cpu.

Reproduction

Run these commands in Bash with Python 3.12.3. The example is CC0.

git clone https://github.com/microsoft/microxcaling.git microxcaling
cd microxcaling
git checkout --detach 7bc41952de394f5cc5e782baf132e7c7542eb4e4
python3.12 -m venv .venv
.venv/bin/python -m pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cpu
.venv/bin/python -m pip install numpy==1.26.4 packaging ninja==1.13.2 hatchling==1.32.4 hatch-vcs==0.5.0 editables
MAX_JOBS=1 .venv/bin/python -m pip install --no-build-isolation --no-deps -e .
OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 .venv/bin/python - <<'PY'
import torch
from mx import linear
from mx.mx_ops import quantize_mx_op
from mx.specs import MxSpecs

torch.set_num_threads(1)
specs = MxSpecs()
specs.update(scale_bits=8, block_size=32, custom_cuda=False)

formats = (
    "fp8_e5m2", "fp8_e4m3", "fp6_e3m2",
    "fp6_e2m3", "fp4_e2m1", "int8",
)
for fmt in formats:
    x = torch.zeros(32, dtype=torch.float16)
    y = quantize_mx_op(x, specs, fmt, axes=-1, round="even")
    print(fmt, "zero NaNs:", y.isnan().sum().item())

x = torch.zeros(32, dtype=torch.float16)
x[0] = 2**-10
y = quantize_mx_op(x, specs, "fp8_e5m2", axes=-1, round="even")
print("E5M2 small:", y[:2].tolist())

specs.update(
    a_elem_format="fp8_e5m2", w_elem_format="fp8_e5m2",
    quantize_backprop=False,
)
output = linear(
    torch.zeros((1, 32), dtype=torch.float16),
    torch.ones((1, 32), dtype=torch.float16),
    mx_specs=specs,
)
print("linear:", output.item())
PY

For the other tested Torch version, use a fresh venv and substitute torch==2.2.0 in the install command. The no-dependency editable install omits unused torchvision and torchaudio, whose dependency mismatch is already reported in issue 31.

Expected and observed

The README documents native FP16 computation and CPU as the numeric reference. PR 28 added native FP16 support.

All-zero blocks should remain zero. For the small E5M2 block, the implemented scale algorithm selects X=2^-25 (E8M0 byte 0x66). The nonzero element is 2^15 (E5M2 code 0x78), so the decoded result is exactly 2^-10. Zero elements remain zero. OCP MX v1.0 §5.1 and Tables 2 and 7 define these finite encodings and products. The scale need not fit the FP16 container. An all-zero block does not require a unique scale.

These encodings are derived expectations for this library's maximum-based scale algorithm. The API returns dequantized tensors, and MX §6.3 permits other scale-selection algorithms.

Case Expected Observed on both Torch versions
Zero block, each of the six formats 32 zeros, no NaNs 32 NaNs
E5M2 block with 2^-10 and 31 zeros Identical finite block 32 NaNs, beginning [nan, nan]
FP16 zero input and unit weights in linear 0.0 nan

Source and scope

mx_ops.py:285 computes 2**shared_exp in FP16. The exact 2^-25 scale rounds to zero before division. Zero blocks also lose the FP32 zero guard at mx_ops.py:78 and underflow the clamped scale. Scale and normalization intermediates need a type that represents these values. No correction was tested.

Zero blocks were checked in all six formats with even, nearest, floor and both FP32-flush settings. The small input is an exact power of two. Wider FP32/BF16 inputs and the FP16 2^-9 threshold control passed. The original C++ CPU numeric code returned the expected values through a CPU-only build adapter. The adapter preserved the numeric code and changed only build plumbing. The stock CUDA extension and GPU execution were not tested. These cases use finite inputs and forward computations.

Found by ChipLab's AI verification agents and independently reproduced.

ChipLab ref: CLAB-2026-623

主要言語
Python
スター
365
フォーク
54
30日以内にマージされた PR はありません

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

microsoft/microxcaling のほかの issue

microsoft/microxcaling の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。