Windows: _get_flatc_path() looks for data/bin/flatc but the wheel ships flatc.exe (WinError 2)
Maintainer antworten meist innerhalb von 1 Tag
Bewertung
- Schwierigkeit
- 2/5
- Geschätzter Aufwand
- 1-3 Stunden
- Anfängerfreundlichkeit
- 83/100
Rechercherichtung
Beginne in exir/_serialize/_flatbuffer.py bei _get_flatc_path() und vergleiche die Ressourcensuche mit der Benennung der Windows-ausführbaren Datei in setup.py. Prüfe die Suche im Windows-Wheel und führe die Reproduktion aus, während das Verzeichnis Scripts der Umgebung aus PATH entfernt ist; fertig ist es, wenn die paketierte flatc.exe gefunden wird und der XNNPACK-Export in diesem Setup abgeschlossen wird.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
🐛 Describe the bug
On Windows, exir/_serialize/_flatbuffer.py::_get_flatc_path() never finds the flatc binary that the wheel ships, so anything that serializes through flatc fails with FileNotFoundError: [WinError 2] whenever the environment's Scripts directory is not on PATH. This is the Windows half of #17000; @nil-is-all asked for it to be filed separately.
Cause. setup.py installs the binary through get_executable_name(), which appends .exe on Windows, so the wheel contains executorch/data/bin/flatc.exe. The lookup asks for the unsuffixed name:
# exir/_serialize/_flatbuffer.py, main at adc713a2e6, line 275
for package, resource_name in (
(__package__, _FLATC_RESOURCE_NAME),
("executorch.data.bin", "flatc"),
):
...
if flatc_resource.is_file():
is_file() is False for flatc on Windows, both candidates miss, and the code falls back to os.getenv("FLATC_EXECUTABLE", "flatc"), a bare name resolved through PATH. The _get_flatc_path() block in the 1.5.1 wheel is identical to main.
Reproduction. Fresh venv, pip install executorch==1.5.1, pip install torch==2.14.0 (CPU), FLATC_EXECUTABLE unset, no other flatc on PATH, and the venv is not activated (the interpreter is called by its full path):
import torch
from executorch.exir import to_edge_transform_and_lower
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
class M(torch.nn.Module):
def __init__(self):
super().__init__()
self.linear = torch.nn.Linear(4, 3)
def forward(self, x):
return torch.relu(self.linear(x))
ep = torch.export.export(M().eval(), (torch.randn(2, 4),))
to_edge_transform_and_lower(ep, partitioner=[XnnpackPartitioner()]).to_executorch()
<venv>\Scripts\python.exe repro.py
...
File "<venv>\Lib\site-packages\executorch\backends\xnnpack\serialization\xnnpack_graph_serialize.py", line 348, in convert_to_flatbuffer
_flatc_compile(d, schema_path, json_path)
File "<venv>\Lib\site-packages\executorch\exir\_serialize\_flatbuffer.py", line 392, in _flatc_compile
_run_flatc(
File "<venv>\Lib\site-packages\executorch\exir\_serialize\_flatbuffer.py", line 339, in _run_flatc
subprocess.run([_get_flatc_path()] + list(args), check=True)
...
FileNotFoundError: [WinError 2] The system cannot find the file specified
The portable (non-delegated) export in the same script succeeds: program serialization goes through _flatbuffer_program, and I confirmed with a subprocess.run spy that to_edge(...).to_executorch() spawns no flatc at all. Only the delegate payload path (here XNNPACK's convert_to_flatbuffer) needs it.
What I measured around the failure, same environment:
executorch/data/bin contents __init__.py, flatc.exe
spawn <bin>\flatc.exe --version OK flatc version 24.3.25
spawn <bin>\flatc --version OK (CreateProcess appends .exe)
spawn flatc --version FileNotFoundError: [WinError 2]
spawn <venv>\Scripts\flatc.exe --version OK (console-script shim from pyproject.toml)
So the binary and subprocess are fine. Only the is_file() check on the unsuffixed name fails, and the bare-name fallback works only when something called flatc is on PATH.
Why it is not seen everywhere. The [project.scripts] entry flatc = "executorch.data.bin:flatc" makes pip create Scripts\flatc.exe, which forwards to the packaged binary. In an activated venv (or any environment whose Scripts is on PATH) the bare-name fallback finds that shim and the export works. Same script, same install, venv Scripts prepended to PATH:
portable : OK
xnnpack : OK
That would also explain why the Windows wheel smoke test (.ci/scripts/wheel/test_windows.py, which runs the same XNNPACK to_executorch() flow) does not catch it; I have not looked at how that job sets up PATH. The failing setups are the ones that call the interpreter without activation: IDE run configurations, Jupyter or other kernels pointing at a venv's python.exe, scheduled tasks, and tools that spawn the interpreter themselves.
I need to correct two statements from my earlier comments on #17000. I wrote that the wheel is only exercised by a shell script that does not run on Windows; that is wrong, the Windows wheel test does run an XNNPACK export, it just runs with Scripts on PATH. And "dies on Windows" was broader than what I can show: the failure needs Scripts to be off PATH.
Workaround today: set FLATC_EXECUTABLE=<site-packages>\executorch\data\bin\flatc.exe, or put the venv's Scripts directory on PATH.
Proposed fix (PR to follow, one line of behaviour): look up flatc.exe on win32, matching what setup.py packages.
("executorch.data.bin", "flatc.exe" if sys.platform == "win32" else "flatc"),
I checked the other places that could resolve a packaged executable by bare name. The wheel ships exactly one executable, flatc.exe. All eight modules that run flatc (xnnpack, vulkan, qualcomm, samsung, native serialization, bundled_program, etdump, flat_tensor) go through _flatc_compile, _flatc_decompile or _run_flatc in this file, so none has its own lookup. The two generate_*.py regeneration scripts and backends/mlx/serialization/generate.py are checkout-side tools that take FLATC_EXECUTABLE or a --flatc path and do not read the wheel. shutil.which callers (for example backends/arm/test/runner_utils.py) already honour PATHEXT. The one other place that looks for the packaged binary by the unsuffixed name is find_executorch_flatc() in backends/arm/scripts/build_executor_runner.sh; it is a bash script for the Arm bare-metal flow and I did not test it on Windows, so I left it alone.
Versions
OS: Microsoft Windows 11 (10.0.26300 64 bit), x86_64
Python: 3.12.10 (64-bit)
executorch==1.5.1 (PyPI win_amd64 wheel), torch==2.14.0+cpu, torchao==0.18.0, numpy==2.5.4
- Vorherrschende Sprache
- Python
- Sterne
- 5.1k
- Forks
- 1.2k
- Ø Merge
- 2 T. 41 Min.
- Gemergte PRs (30 T.)
- 683
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Hat eine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus pytorch/executorch
-
Portable remainder kernel: wrong sign for integers and float64 rounded to floatEvtl. vergeben @JakeStevens hat das vor 8 Tagen übernommen. Offenbug module: kernels triaged
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 82/100
pytorch/executorch#23682 · 2 zugewiesene Personen ·
Maintainer antworten meist innerhalb von 1 Tag
-
enhancement triaged
Schwierigkeit 2/5 Ein halber Tag Anfängerfreundlichkeit 68/100
pytorch/executorch#21640 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 40/100
pytorch/executorch#23955 ·
Maintainer antworten meist innerhalb von 1 Tag
-
[QNN] 16a4w_block stride-2 conv + ReLU fails HTP graph prepare on SM8850 (Relu16bParamsPlaceholder)Evtl. vergeben @psiddh hat das vor 4 Tagen übernommen. Offenbug module: qnn partner: qualcomm
pytorch/executorch#23922 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
-
Cortex-M backend replaces channels-first dw conv with channels-last dw conv without handling the changed layout, fails at runtimeEvtl. vergeben @rahimuddin140 hat das vor 4 Tagen übernommen. Offenbug module: microcontrollers partner: arm
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 38/100
pytorch/executorch#23812 · 2 Kommentare · 1 Reaktion ·
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in pytorch/executorch
Ähnliche Issues
-
[Bug]: `hermes plugins update` clones an entire monorepo twice - `subdir` is resolved but dropped at both catalog-update call sitesEvtl. vergeben @HootRock hat das heute übernommen. Offencomp/cli P2 type/perf
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
NousResearch/hermes-agent#142417 · 2 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug enhancement python
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 85/100
DewaldOosthuizen/tech-cheat-sheets-and-notes#392 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 70/100
Maintainer antworten meist innerhalb von 1 Tag
-
bug p2
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 76/100
Maintainer antworten meist innerhalb von 1 Tag
-
Restore postgresql_vm_test_charm actions.yaml change (removed in canonical/postgresql-operator#2054)Offen
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 85/100
canonical/postgresql-single-kernel-library#390 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 2 Tagen