Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

[ET-VK][Adreno] Non-deterministic conv2d output for c_out in {168,176,184,192}; clean on Mali

オープン
  • 何か問題がありますか？
#21,938 コメント 2 件 リアクション 0 件 担当者 1 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@SS-JIA がすでに取り組んでいます。

2026年8月24日 から。

評価

この issue はまだ評価されていません。

説明

module: vulkan
🐛 Describe the bug

On Adreno, a single Conv2d lowered to the Vulkan delegate produces a different result on almost every execution, for a specific band of output channel counts. Identical .pte, identical input, same already-encoded command buffer replayed in one process.

Probe: Conv2d(48, c_out, 3, padding=1, bias=False) on a (1, 48, 64, 64) input, 200 in-process replays per run, order rotated across runs, cool device.

c_out excess distinct outputs replays runs with any deviation
144 0 1600 0 / 8
160 0 1600 0 / 8
168 1142 1800 9 / 9
176 427 800 4 / 4
184 770 1000 5 / 5
192 772 1200 6 / 6
200 0 1400 0 / 7
224 0 600 0 / 3

Perfect separation. Every run of every c_out in {168, 176, 184, 192} deviates; no run of {144, 160, 200, 224} deviates across 5200 replays.

The band lies inside the im2col/GEMM convolution path: should_use_conv2d_im2col() routes any conv with c_out >= kIm2colMinCOut (128) there. Forcing that function to return false, so the direct conv2d shader is used, makes all of these deterministic. Note 160 and 200 are also in that path and are clean, so it is a size band inside the path, not the threshold itself.

This is very likely an Adreno driver or hardware problem, not an ExecuTorch bug

Three independent checks point away from ExecuTorch:

  1. A different GPU vendor is clean. The same binaries and the same .pte files on a Galaxy S10+ (SM-G975F, Exynos 9820, Mali-G76, Android 12): all eight c_out values, 200 replays each, zero deviations. This is the strongest signal, because should_use_conv2d_im2col() returns graph.device_is_mali() || c_out >= kIm2colMinCOut, so on Mali every conv takes the im2col path, including 144 and 160 which do not on Adreno. If the im2col shader itself were wrong, Mali should have been the worse device.
    Confirmed at the application level as well, not just with executor_runner: a demo app built from the stock Vulkan backend runs both object detection (YOLO26) and image classification (EfficientNetV2-S) on the S10 with correct and stable results.

Separately, and worth knowing for anyone weighing the Vulkan delegate on Mali: on this S10 the Vulkan delegate is close to 2x slower than the XNNPACK equivalent for the same models. That is the opposite of the Adreno device, where Vulkan is clearly faster. The S10 is a 2019 part so this is not necessarily representative of current Mali hardware, but it does mean the delegate is fast on the GPU where it is incorrect and slow on the GPU where it is correct.

  1. Output is numerically correct on Mali, not merely stable. Exporting a Vulkan .pte and a CPU .pte from the same model instance and comparing both against the eager reference gives 9.9e-07 (Vulkan) and 3.6e-07 (CPU), i.e. fp32 rounding.
  2. Khronos synchronization validation reports zero hazards on the failing workload (layer injected into a debuggable app via gpu_debug_layers, sync validation confirmed active by the layer's own settings message). No SYNC-HAZARD, no VUID errors.

So the return false workaround most likely helps by avoiding whatever the Adreno driver miscompiles, rather than by avoiding an incorrect shader.

Reproducing

Counting distinct md5 of executor_runner stdout across repeated process launches is a poor measure: it is noisy, and on this device the failure rate of the related issue #21939 depends strongly on GPU power state. Patch executor_runner to dump outputs on every iteration of --num_executions (stock writes only the last), then count distinct outputs across in-process replays. This one is stable and reproduces at 215 to 262 MHz on a cool device, so unlike #21939 it is not sensitive to clock.

Possibly the same underlying defect as #21939, which is also Adreno-only but low-rate and power-state dependent.

Versions

ExecuTorch 1.4.1. Failing: Galaxy S26 Ultra (SM-S948B), Adreno 840v2. Clean: Galaxy S10+ (SM-G975F), Mali-G76.

主要言語
Python
スター
5.1k
フォーク
1.2k
平均マージ
2日 51分
マージ済み PR（30日）
712

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

pytorch/executorch のほかの issue

pytorch/executorch の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。