[Bug][Relax][ONNX] FLOAT16 DequantizeLinear saturates overflow instead of producing infinity

Start with the Relax legalizer for dequantize in python/tvm/relax/transform/legalize_ops/qdq.py, where clip_cast clips the intermediate to the output dtype's finite range before casting to FLOAT16. Run the repro script's scale=40000 case to confirm the wrong [40000, 65504] result. Done when the ONNX FLOAT16 DequantizeLinear output matches the reference [40000, inf] while the scale=20000 control still passes.

This example needs no NaN, zero scale, or infinite source input.

All source inputs are finite: UINT8 values [1,2], FLOAT16 scale 40000, and zero point 0. ONNX DequantizeLinear should produce [40000,+Inf] after FLOAT16 conversion, but TVM produces [40000,65504]. With scale 20000, both results are [20000,40000].

A runnable case

Save the code below as repro.py. With an LLVM-enabled TVM installation and the dependencies listed at the end, run OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 python repro.py.

from pathlib import Path
import hashlib
import numpy as np
import onnx
import onnxruntime as ort
import tvm
from onnx import TensorProto as T, helper as h, numpy_helper as nh
from onnx.reference import ReferenceEvaluator
from tvm import relax
from tvm.relax.frontend.onnx import from_onnx


def make(nodes, inputs, outputs, initializers=(), opset=18):
    model = h.make_model(h.make_graph(nodes, "repro", inputs, outputs, list(initializers)),
                         opset_imports=[h.make_opsetid("", opset)], ir_version=10)
    onnx.checker.check_model(model, full_check=True)
    return model


def tvm_run(model, feeds):
    mod = from_onnx(model, keep_params_in_input=False)
    names = [value.name for value in model.graph.input]
    executable = relax.build(mod, target="llvm", relax_pipeline="default")
    result = relax.VirtualMachine(executable, tvm.cpu())["main"](
        *[tvm.runtime.tensor(feeds[name]) for name in names])
    return [result.numpy()] if hasattr(result, "numpy") else [v.numpy() for v in result]


def save(label, model, feeds):
    folder = Path("artifacts") / label
    folder.mkdir(parents=True, exist_ok=True)
    onnx.checker.check_model(model, full_check=True)
    onnx.save(model, folder / "model.onnx")
    np.savez(folder / "inputs.npz", **feeds)
    print(label, "model_sha256", hashlib.sha256((folder / "model.onnx").read_bytes()).hexdigest())
    print(label, "inputs_sha256", hashlib.sha256((folder / "inputs.npz").read_bytes()).hexdigest())


def reference(model, feeds, expected, check_ort=True):
    outputs = ReferenceEvaluator(model).run(None, feeds)
    for a, b in zip(outputs, expected, strict=True):
        assert a.shape == b.shape and a.dtype == b.dtype, (a.shape, a.dtype, b.shape, b.dtype)
        np.testing.assert_allclose(a, b, rtol=1e-5, atol=1e-6)
    if check_ort:
        opts = ort.SessionOptions()
        opts.intra_op_num_threads = opts.inter_op_num_threads = 1
        opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
        outputs = ort.InferenceSession(model.SerializeToString(), opts,
            providers=["CPUExecutionProvider"]).run(None, feeds)
        for a, b in zip(outputs, expected, strict=True):
            assert a.shape == b.shape and a.dtype == b.dtype
            np.testing.assert_allclose(a, b, rtol=1e-5, atol=1e-6)


def error_line(error):
    return type(error).__name__ + ": " + str(error).splitlines()[-1]


def same(actual, expected):
    assert actual.shape == expected.shape and actual.dtype == expected.dtype
    np.testing.assert_allclose(actual, expected, rtol=1e-5, atol=1e-6)


x = np.array([1, 2], dtype="uint8")
for scale_value in (40000, 20000):
    scale = np.array(scale_value, dtype="float16")
    model = make([h.make_node("DequantizeLinear", ["x", "s", "z"], ["y"])],
        [h.make_tensor_value_info("x", T.UINT8, [2])],
        [h.make_tensor_value_info("y", T.FLOAT16, [2])],
        [nh.from_array(scale, "s"), nh.from_array(np.array(0, dtype="uint8"), "z")], opset=19)
    with np.errstate(over="ignore"):
        expected = (x.astype("float32") * np.float32(scale)).astype("float16")
    save(str(scale_value), model, {"x": x})
    reference(model, {"x": x}, [expected])
    actual = tvm_run(model, {"x": x})[0]
    print("scale", scale_value, "expected", expected.tolist(), "TVM", actual.tolist())
    if scale_value == 40000:
        assert actual.tolist() == [40000.0, 65504.0] and np.isposinf(expected[1])
    else:
        same(actual, expected)
        print("finite-result control passed")

What I see

scale 40000 expected [40000.0, inf] TVM [40000.0, 65504.0]
scale 20000 expected [20000.0, 40000.0] TVM [20000.0, 40000.0]
finite-result control passed

The expected/actual distinction is: scale=40000: expected [40000,inf], TVM [40000,65504]. scale=20000 finite-result control passes exactly.

Why I think this is the relevant boundary

DequantizeLinear-19 uses (x-zero_point)*scale and the scale's output type. It does not request finite saturation. NumPy's FLOAT32 multiplication followed by FLOAT16 conversion, ONNX ReferenceEvaluator and unoptimized ORT CPU agree on [40000,+Inf].

The ONNX converter lowers to relax.dequantize. Its legalizer calls clip_cast, clipping the intermediate to the output dtype's finite extrema before casting. This report concerns adapting ONNX semantics correctly; it does not assume the generic Relax saturating contract should be changed for all frontends.

Scope of this example

This is dequantization's floating output saturation, not quantization's integer clipping bounds or a zero/NaN scale ambiguity. The minimal witness was constructed during causal follow-up of saved tests; it is not presented as a directly generated historical alarm. No matching report was found in the targeted search.

This self-contained script is a newly constructed minimal diagnostic from the same graph-testing investigation, not a byte-identical replay of an archived model. It needs only TVM, ONNX, NumPy and ONNX Runtime, downloads no files, and writes its generated models and inputs to artifacts/. The assertions check both the observed failure/wrong result and the normal controls; exit code zero means that diagnostic reproduced. No global novelty or current-main execution claim is made.

Reproduction setup

Linux x86_64 CPU/LLVM; Python 3.12.3; TVM 0.28.dev0; LLVM 18.1.3; ONNX 1.22.0; NumPy 2.5.4; ONNX Runtime 1.29.0 (CPUExecutionProvider).

The installed TVM binary's exact build source revision is UNKNOWN. Source links are a separate static inspection of main at 70ec7d5bf20326ef301ee3f06eb6aed51760331d; that revision was not built or executed for this report.

Loaded binary SHA256 identities:

• libtvm_ffi.so: 12072d0ccd9d5cb2c6dac80d42f4217ab50dbf795db03197000614a7a770a816
• libtvm_compiler.so: 574a573fe994e5eda7352ddd8d685e64e45c7934bdeeaab57a5a272a77792ab1
• libtvm_runtime.so: 4175916fc0ce6266c912629d2255094b94087425c6ab53ffd5e004d4c292c828
• libtvm_runtime_extra.so: 79c93351fca12dd7851964890cea7da65dd2b2e0e64e5c28bfed56c878b70817

The code and inputs are small enough to recreate without an attachment. For an exact file check, repro.py has SHA256 5d1212c5574f55563e2bd2e8dd65c0bcc2e6fbc240d07b4e4f02a28f0e387385. Generated model/input checksums from this run:

• artifacts/20000/inputs.npz: 2b032cef3026fbe3f100a6cbe99f1912acf36fe66fbb85f89949391df0665828
• artifacts/20000/model.onnx: 525ec7c2351750ca04338e5782f351c3de43a84150da92ffeb08b078d492c4ed
• artifacts/40000/inputs.npz: 2b032cef3026fbe3f100a6cbe99f1912acf36fe66fbb85f89949391df0665828
• artifacts/40000/model.onnx: 2b70b8d1e47671119ca7920643ad16a7773beadbd5090d6d2070f4b98056bd67

I can follow up with a focused regression test and adjust the example if another supported lowering path is preferred.