Double rounding fix is (still) wrong in Windows
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
Research direction
Start at fxx.ml line 285 and reproduce the minimal f32.const case on Windows, comparing it with the hexadecimal-literal case. Trace how the platform-dependent %.25g formatting affects conversion and verify that the decimal value rounds to 0x1.000002p-50 without relying on msvcrt precision limits.
Written by the indexing model from the issue text.
Description
Found during the development of WAH with a binary script converted from const.wast. Minimal reproduction:
(module (func (export "f") (result f32) (f32.const +8.8817847263968443574e-16)))
(assert_return (invoke "f") (f32.const +0x1.000002p-50))
(Note that a similar construction in hexadecimal literals, (f32.const +0x1.00000100000000001p-50), correctly rounds to 0x1.000002p-50.)
The value in question is almost in the middle of two consecutive binary32 fp number 0x1.0p-50 and 0x1.000002p-50, but it is not the exact middle and in fact very slightly closer to 0x1.000002p-50:
d=8.88178419700125232338905334472656250000000000e-16
+0.00000052939559203401094665527343750000000000e-16
----------------------------------------------------
n=8.88178472639684435740000000000000000000000000e-16
+0.000000529395592033864477180129688349552452..e-16
----------------------------------------------------
u=8.881785255792436391264477180129688349552452..e-16
The root cause is the inaccuracy of fxx.ml line 285. String.length s here is 25 (20 significant digits + . + e-16), so the format string is %.25g, but %g is platform-dependent! (source) In my experience, msvcrt's printf notably does not give any more significant digits than the type's own limit (even %.100f gives ~18 significant digits followed by zeroes), so %.25g actually doesn't make any difference with msvcrt. The correct approach would be to use an actual bigint (scaled by, say, 5^1100 to elide fractional digits).
- Dominant language
- WebAssembly
- Stars
- 3.5k
- Forks
- 539
- Avg merge
- 10h 30m
- Merged PRs (30d)
- 15
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from WebAssembly/spec
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
WebAssembly/spec#2269 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 54/100
WebAssembly/spec#2265 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
WebAssembly/spec#2258 · 4 comments · 1 reaction ·
Maintainers usually reply within 1 day
-
[js-api] A mutable global import allocates a const global before LinkErrorPossibly taken @chicoxyzzy claimed this 4 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 58/100
WebAssembly/spec#2253 ·
Maintainers usually reply within 1 day
-
[spectec] Wasm 1.0: `$instantiate` missing premisesPossibly taken @rossberg claimed this 30 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 68/100
WebAssembly/spec#2245 ·
Maintainers usually reply within 1 day
All issues in WebAssembly/spec
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Default-import note suggests `import * as process` for velt:process, which does not name the builtinOpen
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
diagnostics good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
category:runtime
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day