ESPHome: build sequence does not stop cleanly when download failed · platformio/platformio-core#5308

(2 comments) (0 reactions) (0 assignees)Python (7,329 stars) (791 forks)batch import

help wantedpackage management

Description

Due to a network issue, I am unable to download some larger files at the momemnt.

This lets me see an unusual error - poor handling of a download failure in the build sequence for an ESPhome device

INFO ESPHome 2025.10.4
INFO Reading configuration /config/esphome/living-room-nspanel.yaml...
WARNING GPIO5 is a strapping PIN and should only be used for I/O with care.
Attaching external pullup/down resistors to strapping pins can cause unexpected failures.
See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins
INFO Detected timezone 'Europe/London'
INFO Generating C++ source...
INFO Compiling app...
Processing living-room-nspanel (board: esp32dev; framework: espidf; platform: https://github.com/pioarduino/platform-espressif32/releases/download/54.03.21-2/platform-espressif32.zip)
--------------------------------------------------------------------------------
INFO Package configuration completed successfully
Tool Manager: Installing https://github.com/pioarduino/esp-idf/releases/download/v5.4.2/esp-idf-v5.4.2.zip
INFO Installing https://github.com/pioarduino/esp-idf/releases/download/v5.4.2/esp-idf-v5.4.2.zip
Downloading  [##############----------------------]   39%  00:00:15
Downloading...
Tool Manager: Error: Please read https://bit.ly/package-manager-ioerror
ERROR Error: Please read https://bit.ly/package-manager-ioerror
ChunkedEncodingError: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/urllib3/response.py", line 779, in _error_catcher
    yield
  File "/usr/local/lib/python3.12/site-packages/urllib3/response.py", line 925, in _raw_read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
urllib3.exceptions.IncompleteRead: IncompleteRead(34568354 bytes read, 52483591 more expected)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/requests/models.py", line 820, in generate
    yield from self.raw.stream(chunk_size, decode_content=True)
  File "/usr/local/lib/python3.12/site-packages/urllib3/response.py", line 1091, in stream
    data = self.read(amt=amt, decode_content=decode_content)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/response.py", line 1008, in read
    data = self._raw_read(amt)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/response.py", line 903, in _raw_read
    with self._error_catcher():
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/usr/local/lib/python3.12/site-packages/urllib3/response.py", line 803, in _error_catcher
    raise ProtocolError(arg, e) from e
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(34568354 bytes read, 52483591 more expected)', IncompleteRead(34568354 bytes read, 52483591 more expected))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/platformio/__main__.py", line 103, in main
    cli()  # pylint: disable=no-value-for-parameter
    ^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/platformio/cli.py", line 85, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/platformio/run/cli.py", line 147, in cli
    process_env(
  File "/usr/local/lib/python3.12/site-packages/platformio/run/cli.py", line 210, in process_env
    ).process()
      ^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/platformio/run/processor.py", line 81, in process
    install_project_env_dependencies(
  File "/usr/local/lib/python3.12/site-packages/platformio/package/commands/install.py", line 132, in install_project_env_dependencies
    _install_project_env_platform(project_env, options),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/platformio/package/commands/install.py", line 149, in _install_project_env_platform
    PlatformPackageManager().install(
  File "/usr/local/lib/python3.12/site-packages/platformio/package/manager/platform.py", line 62, in install
    p.install_required_packages(force=force)
  File "/usr/local/lib/python3.12/site-packages/platformio/platform/_packages.py", line 76, in install_required_packages
    self.install_package(name, force=force)
  File "/usr/local/lib/python3.12/site-packages/platformio/platform/_packages.py", line 70, in install_package
    return self.pm.install(spec or self.get_package_spec(name), force=force)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/platformio/package/manager/_install.py", line 47, in install
    pkg = self._install(spec, skip_dependencies=skip_dependencies, force=force)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/platformio/package/manager/_install.py", line 97, in _install
    pkg = self.install_from_uri(spec.uri, spec)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/platformio/package/manager/_install.py", line 189, in install_from_uri
    dl_path = self.download(uri, checksum)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/platformio/package/manager/_download.py", line 88, in download
    raise exc
  File "/usr/local/lib/python3.12/site-packages/platformio/package/manager/_download.py", line 71, in download
    fd.start(with_progress=with_progress, silent=silent)
  File "/usr/local/lib/python3.12/site-packages/platformio/package/download.py", line 110, in start
    for chunk in pb:
                 ^^
  File "/usr/local/lib/python3.12/site-packages/click/_termui_impl.py", line 344, in generator
    for rv in self.iter:
              ^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests/models.py", line 822, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(34568354 bytes read, 52483591 more expected)', IncompleteRead(34568354 bytes read, 52483591 more expected))

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
  `python -m pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
  https://docs.platformio.org/page/faq/index.html

* Report this problem to the developers
  https://github.com/platformio/platformio-core/issues

============================================================

Contributor guide

Tech stack: python
Domain: build systemtoolingdevops
Issue type: bug
Difficulty: 3
Estimated time: half day
Activity status: fresh
Clarity: clear
Prerequisites: Pythonunderstanding of platformio package download
Newbie friendliness: 40
Research direction: Investigate the download failure handling in platformio/package/manager/ download.py and platformio/package/download.py. The issue shows that an IncompleteRead exception leads to a crash without stopping the build sequence cleanly. Look at how exceptions are caught in the download process and whether the build should abort early on download failure. Consider adding a check to ensure that if a download fails, the build is halted with a clear error message instead of continuing.