quantize_bf16 silently eliminates a scalar Mul with a literal near 1.0 (e.g. 1.0507)

Aperta
  • Qualcosa non va?
#5,491 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 3 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
68/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cpp, python
Ambito
backend, machine-learning

Direzione di ricerca

Start by reading the BF16 quantization and simplification code that handles scalar Mul operations, then use the self-contained ONNX graph in the issue to reproduce the behavior on the ref target. Check why the literal 1.0507 causes Mul to disappear while 1.2 remains; done when non-unit multipliers are preserved and the regression is covered by a test.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

migraphx.quantize_bf16() removes a scalar Mul whose literal is sufficiently close
to 1.0. The multiply is simply absent from the compiled program — there is no
warning and no diagnostic. FP32 compilation of the same graph is unaffected; the
elimination happens only on the BF16 path.

Versions

Reproduced on MIGraphX 2.17.0 and on 2.15.0 (2.15.0.20250912-17-221-gdd11a7555,
ROCm 7.2.4), on both the CPU ref target and a gfx1151 GPU target.

Reproducer (self-contained, no AMD GPU required)

Graph: output = Mul(Relu(x), k) — a single scalar multiply on a Relu output.

import numpy as np, onnx, migraphx
from onnx import helper, numpy_helper, TensorProto

def save_mul_graph(k, path):
    xi = helper.make_tensor_value_info("input",  TensorProto.FLOAT, [1, 1])
    yo = helper.make_tensor_value_info("output", TensorProto.FLOAT, [1, 1])
    nodes = [helper.make_node("Relu", ["input"], ["r"]),
             helper.make_node("Mul",  ["r", "k"], ["output"])]
    init  = [numpy_helper.from_array(np.array([k], dtype=np.float32), "k")]
    m = helper.make_model(helper.make_graph(nodes, "mul", [xi], [yo], initializer=init),
                          opset_imports=[helper.make_opsetid("", 13)])
    m.ir_version = 7
    onnx.save(m, path)

for k in [1.0507, 1.05, 1.2, 1.5, 2.0, 0.5]:
    save_mul_graph(k, "mul_near_one.onnx")
    prog = migraphx.parse_onnx("mul_near_one.onnx")
    migraphx.quantize_bf16(prog)
    prog.compile(migraphx.get_target("ref"))   # CPU ref; "gpu" reproduces identically
    out = np.array(prog.run({"input": np.array([[4.0]], dtype=np.float32)})[-1])
    dropped = "DROPPED" if abs(out.item() - 4.0) < 1e-3 else "ok"
    print(f"k={k:<7} got={out.item():8.4f}  expected={4.0*k:8.4f}  {dropped}")
k 1.0507 1.05 1.2 1.5 2.0 0.5
after quantize_bf16() dropped dropped ok ok ok ok

When dropped, the output equals the input exactly (got == 4.0), i.e. the Mul op is
fully eliminated rather than rounded.

Expected behaviour

A Mul by any literal other than exactly 1.0 must be preserved. If the pass is
deliberately folding multiplies whose BF16-rounded value is 1.0, note that 1.0507
does not round to 1.0 in BF16 (BF16 has 8 mantissa bits; the nearest BF16 value
to 1.0507 is 1.046875). The elimination therefore looks like a tolerance-comparison
bug rather than a legitimate fold.

Suggested investigation

Check whether the BF16 pass's simplification predicate compares the literal against
1.0 with an absolute tolerance (something on the order of 0.05–0.06 would explain
the observed 1.05-pass / 1.2-fail boundary) instead of testing for exact equality — or
equality after BF16 rounding.

Context (how we hit it)

This interacts badly with a sibling parse_selu defect. The natural workaround for
that bug is an explicit Where(x > 0, gamma*x, gamma*alpha*(exp(min(x,0)) - 1)).
SELU's gamma = 1.0507 lands inside the elimination window, so on the BF16 path
the positive branch silently lost its scale while the negative branch (constant
gamma*alpha = 1.758, outside the window) kept it — a 4.8 % relative rescale between
the two branches. Re-associating the positive branch as x + (gamma-1)*x (literal
0.0507, clear of the window) is what finally dodged it.

Lingua principale
C++
Stelle
338
Fork
161
Merge medio
5g 21h
PR unite (30g)
45

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ROCm/AMDMIGraphX

Tutte le issue di ROCm/AMDMIGraphX

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.