Windows: _get_flatc_path() looks for data/bin/flatc but the wheel ships flatc.exe (WinError 2)
Los mantenedores suelen responder en 1 día
Evaluación
- Dificultad
- 2/5
- Tiempo estimado
- 1-3 horas
- Aptitud para principiantes
- 83/100
Línea de trabajo
Empieza en exir/_serialize/_flatbuffer.py, en _get_flatc_path(), y compara la búsqueda del recurso con el nombre del ejecutable de Windows en setup.py. Comprueba la búsqueda en el wheel de Windows y ejecuta la reproducción con el directorio Scripts del entorno fuera de PATH; se considera terminado cuando se encuentra el flatc.exe empaquetado y se completa la exportación de XNNPACK en esa configuración.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
🐛 Describe the bug
On Windows, exir/_serialize/_flatbuffer.py::_get_flatc_path() never finds the flatc binary that the wheel ships, so anything that serializes through flatc fails with FileNotFoundError: [WinError 2] whenever the environment's Scripts directory is not on PATH. This is the Windows half of #17000; @nil-is-all asked for it to be filed separately.
Cause. setup.py installs the binary through get_executable_name(), which appends .exe on Windows, so the wheel contains executorch/data/bin/flatc.exe. The lookup asks for the unsuffixed name:
# exir/_serialize/_flatbuffer.py, main at adc713a2e6, line 275
for package, resource_name in (
(__package__, _FLATC_RESOURCE_NAME),
("executorch.data.bin", "flatc"),
):
...
if flatc_resource.is_file():
is_file() is False for flatc on Windows, both candidates miss, and the code falls back to os.getenv("FLATC_EXECUTABLE", "flatc"), a bare name resolved through PATH. The _get_flatc_path() block in the 1.5.1 wheel is identical to main.
Reproduction. Fresh venv, pip install executorch==1.5.1, pip install torch==2.14.0 (CPU), FLATC_EXECUTABLE unset, no other flatc on PATH, and the venv is not activated (the interpreter is called by its full path):
import torch
from executorch.exir import to_edge_transform_and_lower
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
class M(torch.nn.Module):
def __init__(self):
super().__init__()
self.linear = torch.nn.Linear(4, 3)
def forward(self, x):
return torch.relu(self.linear(x))
ep = torch.export.export(M().eval(), (torch.randn(2, 4),))
to_edge_transform_and_lower(ep, partitioner=[XnnpackPartitioner()]).to_executorch()
<venv>\Scripts\python.exe repro.py
...
File "<venv>\Lib\site-packages\executorch\backends\xnnpack\serialization\xnnpack_graph_serialize.py", line 348, in convert_to_flatbuffer
_flatc_compile(d, schema_path, json_path)
File "<venv>\Lib\site-packages\executorch\exir\_serialize\_flatbuffer.py", line 392, in _flatc_compile
_run_flatc(
File "<venv>\Lib\site-packages\executorch\exir\_serialize\_flatbuffer.py", line 339, in _run_flatc
subprocess.run([_get_flatc_path()] + list(args), check=True)
...
FileNotFoundError: [WinError 2] The system cannot find the file specified
The portable (non-delegated) export in the same script succeeds: program serialization goes through _flatbuffer_program, and I confirmed with a subprocess.run spy that to_edge(...).to_executorch() spawns no flatc at all. Only the delegate payload path (here XNNPACK's convert_to_flatbuffer) needs it.
What I measured around the failure, same environment:
executorch/data/bin contents __init__.py, flatc.exe
spawn <bin>\flatc.exe --version OK flatc version 24.3.25
spawn <bin>\flatc --version OK (CreateProcess appends .exe)
spawn flatc --version FileNotFoundError: [WinError 2]
spawn <venv>\Scripts\flatc.exe --version OK (console-script shim from pyproject.toml)
So the binary and subprocess are fine. Only the is_file() check on the unsuffixed name fails, and the bare-name fallback works only when something called flatc is on PATH.
Why it is not seen everywhere. The [project.scripts] entry flatc = "executorch.data.bin:flatc" makes pip create Scripts\flatc.exe, which forwards to the packaged binary. In an activated venv (or any environment whose Scripts is on PATH) the bare-name fallback finds that shim and the export works. Same script, same install, venv Scripts prepended to PATH:
portable : OK
xnnpack : OK
That would also explain why the Windows wheel smoke test (.ci/scripts/wheel/test_windows.py, which runs the same XNNPACK to_executorch() flow) does not catch it; I have not looked at how that job sets up PATH. The failing setups are the ones that call the interpreter without activation: IDE run configurations, Jupyter or other kernels pointing at a venv's python.exe, scheduled tasks, and tools that spawn the interpreter themselves.
I need to correct two statements from my earlier comments on #17000. I wrote that the wheel is only exercised by a shell script that does not run on Windows; that is wrong, the Windows wheel test does run an XNNPACK export, it just runs with Scripts on PATH. And "dies on Windows" was broader than what I can show: the failure needs Scripts to be off PATH.
Workaround today: set FLATC_EXECUTABLE=<site-packages>\executorch\data\bin\flatc.exe, or put the venv's Scripts directory on PATH.
Proposed fix (PR to follow, one line of behaviour): look up flatc.exe on win32, matching what setup.py packages.
("executorch.data.bin", "flatc.exe" if sys.platform == "win32" else "flatc"),
I checked the other places that could resolve a packaged executable by bare name. The wheel ships exactly one executable, flatc.exe. All eight modules that run flatc (xnnpack, vulkan, qualcomm, samsung, native serialization, bundled_program, etdump, flat_tensor) go through _flatc_compile, _flatc_decompile or _run_flatc in this file, so none has its own lookup. The two generate_*.py regeneration scripts and backends/mlx/serialization/generate.py are checkout-side tools that take FLATC_EXECUTABLE or a --flatc path and do not read the wheel. shutil.which callers (for example backends/arm/test/runner_utils.py) already honour PATHEXT. The one other place that looks for the packaged binary by the unsuffixed name is find_executorch_flatc() in backends/arm/scripts/build_executor_runner.sh; it is a bash script for the Arm bare-metal flow and I did not test it on Windows, so I left it alone.
Versions
OS: Microsoft Windows 11 (10.0.26300 64 bit), x86_64
Python: 3.12.10 (64-bit)
executorch==1.5.1 (PyPI win_amd64 wheel), torch==2.14.0+cpu, torchao==0.18.0, numpy==2.5.4
- Lenguaje dominante
- Python
- Estrellas
- 5.1k
- Forks
- 1.2k
- Merge medio
- 2 d 41 min
- PR fusionados (30 d)
- 683
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de pytorch/executorch
-
Portable remainder kernel: wrong sign for integers and float64 rounded to floatPosiblemente ocupada @JakeStevens la tomó hace 7 días. Abiertobug module: kernels triaged
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
pytorch/executorch#23682 · 2 asignados ·
Los mantenedores suelen responder en 1 día
-
enhancement triaged
Dificultad 2/5 Medio día Aptitud para principiantes 68/100
pytorch/executorch#21640 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 4/5 3-5 días Aptitud para principiantes 40/100
pytorch/executorch#23955 ·
Los mantenedores suelen responder en 1 día
-
[QNN] 16a4w_block stride-2 conv + ReLU fails HTP graph prepare on SM8850 (Relu16bParamsPlaceholder)Posiblemente ocupada @psiddh la tomó hace 3 días. Abiertobug module: qnn partner: qualcomm
pytorch/executorch#23922 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Cortex-M backend replaces channels-first dw conv with channels-last dw conv without handling the changed layout, fails at runtimePosiblemente ocupada @rahimuddin140 la tomó hace 3 días. Abiertobug module: microcontrollers partner: arm
Dificultad 4/5 3-5 días Aptitud para principiantes 38/100
pytorch/executorch#23812 · 2 comentarios · 1 reacción ·
Los mantenedores suelen responder en 1 día
Todos los issues de pytorch/executorch
Issues similares
-
Edit tool converts CRLF files to LFPosiblemente ocupada @RaphaelFakhri la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
huggingface/tau#774 ·
Los mantenedores suelen responder en 1 día
-
show ignores the row limit when a column or table name contains "limit"Posiblemente ocupada @RaphaelFakhri la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
Los mantenedores suelen responder en 1 día
-
Agents panel: Codex token stats always empty because Codex now writes `*.jsonl.zst` sessionsAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
Los mantenedores suelen responder en 2 días
-
documentation good first issue
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
eclipse-score/inc_someip_gateway#356 ·
Los mantenedores suelen responder en 2 días