Linux ARM64 packaging omits worker modules and generates protos incompatible with bundled runtimes
Maintainer antworten meist innerhalb von 4 Tagen
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 48/100
- Issue-Typ
- Bug
- Klarheit
- Klar beschrieben
- Aktivitätsstatus
- Aktiv
- Tech-Stack
- grpc, python
- Bereich
- backend, build-system, testing
Rechercherichtung
Start with eng/pack/scripts/nix_arm64_deps.sh and eng/pack/scripts/rc_nix_arm64_deps.sh to trace how worker files and generated protos enter the ARM64 payload. Then read eng/pack/templates/nix_arm64_env_gen.yml and inspect the packaged Python worker and runtime dependency versions. Done means both packaging paths include complete worker modules, compatible generated code, and validation that catches missing files or import failures.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
Summary
The Linux ARM64 packaging path produces incomplete Python worker payloads. Core Tools 4.14.0–4.16.0 ship worker directories containing generated protos but missing the executable worker modules.
Downstream report: https://github.com/Azure/azure-functions-core-tools/issues/5772
Affected versions
| Core Tools release | Python worker dependency | Worker modules |
|---|---|---|
| 4.13.0 | 4.44.0 | Present |
| 4.14.0 | 4.45.1 | Missing |
| 4.15.0–4.16.0 | 4.46.0 | Missing |
Inspection of 28 published Core Tools ZIP archives (seven OS/architecture combinations across these four releases) confirms the missing modules across every bundled Python version, 3.9–3.14, on Linux ARM64. The equivalent Linux x64, macOS x64/ARM64, and Windows x64/x86 archives retain the worker entry modules and dispatcher. Windows ARM64 does not bundle the Python worker and is a separate, pre-existing limitation.
These findings are based on published archive inspection and source tracing; local execution on ARM64 was reported in the downstream issue, not independently reproduced during this investigation.
Root cause
The ARM64 packaging rewrite in https://github.com/Azure/azure-functions-python-worker/pull/1845 (commit b991f234f0decd481544c52ac06e3b82b6ec3b4b) replaced installation of the local worker project with dependency downloads in eng/pack/scripts/nix_arm64_deps.sh and eng/pack/scripts/rc_nix_arm64_deps.sh:
pip download \
--platform manylinux_2_17_aarch64 \
--only-binary=:all: \
--dest /tmp/arm64_wheels \
.
The subsequent loop extracts only downloaded wheels. pip download . resolves/downloads dependencies but does not build/save the local source directory itself as a wheel. Pip's RequirementPreparer.save_linked_requirement explicitly skips copying a non-editable local directory. Therefore the worker package itself never reaches deps.
Later steps copy back only generated protos/ and, for the older worker, _vendored/. This matches the published incomplete payloads:
- Python 3.9–3.12 lack
azure_functions_worker/main.py,dispatcher.py, and other worker modules. - Python 3.13–3.14 lack
proxy_worker/start_worker.py,dispatcher.py, and other worker modules. - No
.pycfiles remain in these worker packages as an executable substitute in the affected Linux ARM64 releases.
Relevant source: https://github.com/Azure/azure-functions-python-worker/blob/4.46.0/eng/pack/scripts/nix_arm64_deps.sh#L24-L84
The launcher worker.py is still present. For Python 3.13/3.14 it imports proxy_worker.start_worker, so startup fails before the worker can connect to the host:
https://github.com/Azure/azure-functions-python-worker/blob/4.46.0/workers/python/proxyV4/worker.py
Additional generated-code/runtime mismatch
The ARM64 scripts install unpinned grpcio-tools into their build environment and generate protos there, independently of the constrained runtime dependencies downloaded for distribution:
Direct inspection of the published Core Tools 4.16.0 Linux ARM64 archive confirmed:
| Python | Generated protobuf version | Bundled protobuf runtime | Generated gRPC requirement |
|---|---|---|---|
| 3.11 | 7.35.0 | 5.29.6 | grpcio >=1.82.1 |
| 3.14 | 7.35.0 | 6.33.6 | grpcio >=1.82.1 |
The downstream reporter additionally records bundled grpcio 1.75.1 for Python 3.14 and the following failures after manually restoring missing worker modules:
VersionError: Detected incompatible Protobuf Gencode/Runtime versions when loading FunctionRpc.proto: gencode 7.35.0 runtime 6.33.6.
RuntimeError: The grpc package installed is at version 1.75.1, but the generated code in FunctionRpc_pb2_grpc.py depends on grpcio>=1.82.1.
Restoring worker source files alone therefore does not fully repair the payload.
Why artifact validation misses this
eng/pack/templates/nix_arm64_env_gen.yml checks the worker directories and generated proto files, but not required executable entry modules or successful worker/proto imports. A worker directory containing only protos can pass these checks.
Reproduction
- Install Core Tools 4.16.0 on Linux ARM64, using the release ZIP or
.deb. - Create a Python Functions app using Python 3.13 or 3.14.
- Run
func start --verbose.
Reported failure:
Azure.Functions.Rpc.Server: python3 exited with code 1 (0x1). ImportError: cannot import name 'start_worker' from 'proxy_worker' (unknown location).
Archive-only verification:
unzip -l Azure.Functions.Cli.linux-arm64.4.16.0.zip \
| grep 'python/3.14/LINUX/Arm64/proxy_worker/' \
| grep -v protos
No worker implementation files are listed.
Expected resolution
Explicitly build/include the local worker package in both ARM64 packaging paths, align code-generation tools with packaged runtime versions, and validate imports/startup using matching ARM64 Python environments. Publish a corrected worker package for Core Tools to consume.
Acceptance criteria
- Complete worker payloads are packaged for every supported Linux ARM64 Python version.
- Worker entry modules and generated protobuf/gRPC modules import successfully using packaged dependencies.
- An ARM64 Python function app starts, indexes, and executes successfully.
- Validation rejects missing worker modules and incompatible generated-code/runtime combinations.
- Both stable and RC ARM64 packaging paths are covered.
- Vorherrschende Sprache
- Python
- Sterne
- 357
- Forks
- 119
- Ø Merge
- 4 T. 8 Std.
- Gemergte PRs (30 T.)
- 5
Entwicklungsumgebung
Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.
- Kein Dockerfile und keine Docker-Compose-Datei
- Hat eine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus Azure/azure-functions-python-worker
-
[Bug] ASGI cookie conversion serializes an absent Domain attribute and breaks __Host- cookiesEvtl. vergeben @tharun06 hat das vor 39 Tagen übernommen. Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 68/100
Azure/azure-functions-python-worker#1902 ·
Maintainer antworten meist innerhalb von 4 Tagen
-
Schwierigkeit 5/5 Über eine Woche Anfängerfreundlichkeit 28/100
Azure/azure-functions-python-worker#1908 · 2 Kommentare ·
Maintainer antworten meist innerhalb von 4 Tagen
-
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 55/100
Azure/azure-functions-python-worker#1906 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 4 Tagen
-
Schwierigkeit 5/5 Über eine Woche Anfängerfreundlichkeit 35/100
Azure/azure-functions-python-worker#1904 ·
Maintainer antworten meist innerhalb von 4 Tagen
-
[Python 3.14 proxy worker] Missing sys.modules["__main__"] breaks multiprocessing spawnEvtl. vergeben @Fire162 hat das vor 17 Tagen übernommen. Offen
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 55/100
Azure/azure-functions-python-worker#1903 ·
Maintainer antworten meist innerhalb von 4 Tagen
Alle Issues in Azure/azure-functions-python-worker
Ähnliche Issues
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 82/100
Maintainer antworten meist innerhalb von 1 Tag
-
namespace operations
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 65/100
EclipseFdn/open-vsx.org#15057 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 76/100
FinanceFlash/unvibecode#349 ·
-
Mission statement: why this is free, in the README, Why This Exists and the wiki HomeEvtl. vergeben @ChiefGyk3D hat das heute übernommen. Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 76/100
ChiefGyk3D/Clouds-Against-Humanity#348 ·
Maintainer antworten meist innerhalb von 1 Tag
-
[Perf] Default --vllm-watermark 0.05 for rollouts (hybrid models re-preempt under KV pressure)Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 74/100
vllm-project/vime#508 ·
Maintainer antworten meist innerhalb von 4 Tagen