bug: read reopens with a zero-size range after the last byte
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 86/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- rust
- Domain
- backend, testing-qa
Research direction
Start in core/layers/retry/src/lib.rs at RetryReadStream::read and inspect the existing mock and test_retry_read. Run the provided bounded-range reproduction, then verify that the read completes without reopening after the last byte and that the relevant retry tests pass.
Written by the indexing model from the issue text.
Description
OpenDAL version or commit
main (53610f89), and 0.58.0 onward
Affected service or component
core: layers/retry, RetryReadStream::read. Service-independent.
Describe the bug
RetryReadStream::read advances the range by what it has read before reopening (core/layers/retry/src/lib.rs:657). If the stream breaks after the last byte of a bounded range, the range is already exhausted, and it reopens with size zero.
Nothing is left to fetch, and a zero-size range has no valid Range header: Display returns Err for it, so any layer that formats the range panics.
Steps to Reproduce
Against the mock already in core/layers/retry/src/lib.rs, which returns the whole body and then a temporary error:
#[tokio::test]
async fn test_reopen_at_eof() -> Result<()> {
let op = Operator::new(MockBuilder::default())?
.layer(LoggingLayer::default())
.layer(RetryLayer::default());
let r = op.reader("retryable_error").await?;
let mut buf = Vec::new();
r.read_into(&mut buf, 0..13).await?; // panics
Ok(())
}
test_retry_read does not panic because it reads unbounded: that range advances to 13-, which formats fine. Only a size of zero fails.
I found this by reading the code, and the reproduction above is that in-repo mock. I have not seen it against a live backend, which would need a connection failure after the last byte of a bounded range.
Actual Behavior
panicked at library/alloc/src/string.rs:2943:14:
a Display implementation returned an error unexpectedly: Error
Without a formatting layer there is no panic, but the request is still sent: is_full() is false for a zero-size range, so S3 for one sets the header (core/services/s3/src/core.rs:585) with the empty string to_header returns.
Expected Behavior
The read ends once the range is exhausted, instead of reopening.
Additional Context
#8084 made to_header return an empty string for such a range instead of panicking, and noted there that an empty header is not right either. This is the other end of it: the range should not be created at all.
Reproduction confirmation
- I reproduced this issue against the affected component. For a service-specific report, I used the actual service rather than only a mock, emulator, or source-code analysis.
Are you willing to submit a PR to fix this bug?
- Yes, I would like to submit a PR.
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 16h 49m
- Merged PRs (30d)
- 112
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/opendal
-
services/cos
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
bug services/azdls
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
bug releases-note/fix services/fs
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
-
bug integrations/dav-server
Difficulty 4/5 3-5 days Newbie friendliness 72/100
-
enhancement services/webdav
Difficulty 3/5 1-2 days Newbie friendliness 68/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
JIT-compiled number -> Decimal conversion silently overflows instead of raising DECIMAL_OVERFLOW Openfuzz
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
ClickHouse/ClickHouse#122114 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
linebender/vello_svg#90 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
state:needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
zed-industries/zed#64742 · 1 comment ·