[BUG] FP4 quantization on CPU rounds ties toward zero while Metal rounds to even
Maintainers usually reply within 2 days
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 74/100
Research direction
Start with fp_quantize in mlx/ops.cpp and compare its CPU tie rounding with the Metal behavior described in the issue. Use the listed FP4 inputs to reproduce the inconsistent results; done means CPU tie cases round consistently to even, matching the stated expected values.
Written by the indexing model from the issue text.
Description
☑️ I understand it is strictly prohibited to use AI to write issues.
Describe the bug
This bug was identified by comparing floating point number conversion results across a few open-source implementations. On CPU, mx.quantize to FP4 rounds some ties towards zero and some to even while the Metal kernel rounds all ties to even. The OCP MX spec says that implementations must at least support round ties to even (other rounding modes are optional). The CPU implementation should at least be self-consistent. Feel free to close if this is expected/intended.
fp_quantize (mlx/ops.cpp)
| input | expected | observed |
|---|---|---|
0.75 (0x3F400000) |
0x2 = 1 |
0x1 = 0.5 |
1.75 (0x3FE00000) |
0x4 = 2 |
0x3 = 1.5 |
3.5 (0x40600000) |
0x6 = 4 |
0x5 = 3 |
-0.75 (0xBF400000) |
0xA = -1 |
0x9 = -0.5 |
-1.75 (0xBFE00000) |
0xC = -2 |
0xB = -1.5 |
-3.5 (0xC0600000) |
0xE = -4 |
0xD = -3 |
Note that some of the ties are correctly rounded to even: (0.25, 1.25, 2.5, 5)
To Reproduce
Call fp_quantize with the input values provided
Expected behavior
See expected column in the table above
Desktop (please complete the following information):
- OS Version: macOS 26.6.2 (Apple M5 Pro)
- Version: mlx 0.32.4
- Dominant language
- C++
- Stars
- 29k
- Forks
- 2.4k
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 111
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ml-explore/mlx
-
[BUG] `mx.qqmm` docs: the Args section names `global_scale`, but the parameter is `global_scale_x`Possibly taken @Redempt1onzzZZ claimed this 1 day ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 92/100
ml-explore/mlx#4815 ·
Maintainers usually reply within 2 days
-
[BUG] Documented signatures of `mx.add` and 27 other ops omit the keyword-only `*`, so `stream` (and `equal_nan`) cannot be passed positionally as documentedPossibly taken @Redempt1onzzZZ claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ml-explore/mlx#4814 ·
Maintainers usually reply within 2 days
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
ml-explore/mlx#4819 ·
Maintainers usually reply within 2 days
-
[BUG] Metal scaled_dot_product_attention is ~300x less accurate for 2-8 query tokens at 33-128 stacked rows since #4723 (sdpa_split)Possibly taken @STRML claimed this 2 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 22/100
ml-explore/mlx#4808 · 1 comment ·
Maintainers usually reply within 2 days
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
ml-explore/mlx#4806 ·
Maintainers usually reply within 2 days
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
colbymchenry/codegraph#3223 ·
Maintainers usually reply within 1 day
-
The `ufw` decoder requires the kernel's monotonic timestamp, so UFW lines read from journald decode without any fieldPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
arduino/ArduinoCore-megaavr#152 ·
-
bug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
microsoft/microsoft-ui-xaml#12515 · 1 comment · 1 reaction ·
Maintainers usually reply within 2 days
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
BlueQuartzSoftware/simplnx#1906 ·
Maintainers usually reply within 4 days