Local range download leaves an output file after source-read failure despite delete_on_failure=True

Open Beginner friendly
  • Is something wrong?
#2,223 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
cloud

Research direction

Read the range-download contract in libcloud/storage/base.py and the local implementation in libcloud/storage/drivers/local.py. Reproduce the missing-source case from the issue, then add a regression test alongside the local driver's existing tests. Done means a failed range download removes the destination when delete_on_failure=True, while the behavior with the flag false remains as intended.

Written by the indexing model from the issue text.

Description

On trunk 8a0ef626, the local driver's range-download method creates a destination file but does not clean it up when opening the source fails, even with delete_on_failure=True.

This reproducer is sequential: obtain the object metadata, explicitly remove the stored file, then try to download. It does not depend on winning a filesystem race.

import os
import tempfile
from libcloud.storage.drivers.local import LocalStorageDriver

with tempfile.TemporaryDirectory() as root:
    driver = LocalStorageDriver(root)
    container = driver.create_container("bucket")
    obj = driver.upload_object_via_stream(iter([b"payload"]), container, "object")
    os.unlink(driver.get_object_cdn_url(obj))
    destination = os.path.join(root, "download")
    try:
        driver.download_object_range(
            obj, destination, start_bytes=0, delete_on_failure=True
        )
    except FileNotFoundError:
        pass
    print(os.path.exists(destination), os.path.getsize(destination))
    # True 0

The public range-download contract describes delete_on_failure as deleting a partially downloaded file when the download was unsuccessful, with hash mismatch/file size given as examples. I expected the newly created empty output to be removed for a source-read failure as well. Otherwise a failed operation leaves a file that can be mistaken for a valid empty download.

The local implementation accepts the flag but never reads it. It opens the destination before consuming the source iterator and has no failure cleanup around that loop.

Checks with unchanged source, Python 3.12/Linux:

  • Range download with delete_on_failure=True: raises FileNotFoundError, leaves a zero-byte output.
  • Range download with delete_on_failure=False: same result.
  • Full download_object with the same missing source: returns False, leaves no output.
  • Successful range [0, 3) before removing the source: returns True, writes b"pay".

No code change yet. This report is limited to failure cleanup for local range downloads; it does not claim a concurrent-write reproduction or a full test-suite pass.

Dominant language
Python
Stars
2.1k
Forks
935
Avg merge
5d 3h
Merged PRs (30d)
15

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/libcloud

All issues in apache/libcloud

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.