Local filesystem writes can retain stale tail data when overwriting shorter objects
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 86/100
Research direction
Read packages/shared/pkg/storage/storage_fs.go, focusing on fsObject.Put, fsObject.StoreFile, and fsObject.getHandle, then run the storage package tests. Add separate regression coverage for shorter overwrites through Put and uncompressed StoreFile; done means Size and reads contain exactly the replacement payload, while compressed StoreFile and non-replacement paths retain their existing behavior.
Written by the indexing model from the issue text.
Description
Problem
The Local FS storage backend opens existing objects with os.O_RDWR|os.O_CREATE but without os.O_TRUNC:
handle, err := os.OpenFile(o.path, os.O_RDWR|os.O_CREATE, 0o644)
fsObject.Put writes from offset zero, so overwriting an existing object with shorter content leaves the old tail in the file. The same behavior affects the uncompressed StoreFile path that uses getHandle(false).
Example:
existing object: abcdefgh
new object: xy
read result: xycdefgh
This makes the Local FS backend inconsistent with object-storage backends, where a put replaces the complete object. It can corrupt local snapshots, cache files, or metadata when an object is rewritten with a smaller payload.
Proposed fix
Use truncation for complete-object writes, either by adding os.O_TRUNC to the write handle or by explicitly truncating and resetting the offset before copying:
handle, err := os.OpenFile(o.path, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o644)
if err != nil {
return nil, err
}
The truncating mode should only be used for complete replacement writes. Read, range-read, append, and multipart/random-offset paths should retain their existing semantics.
Tests
Please add two separate regression tests:
-
Putpath:- Write a longer payload to an object.
- Overwrite it with a shorter payload using
Put. - Verify
Sizeequals the new payload length. - Verify reading returns exactly the new payload, with no stale suffix.
-
Direct, uncompressed
StoreFilepath:- Store a longer source file into an object.
- Overwrite it with a shorter source file using uncompressed
StoreFile. - Verify
Sizeequals the new source file length. - Verify reading returns exactly the new source content, with no stale suffix.
The compressed StoreFile path is a separate implementation and should not be conflated with this regression.
Relevant code:
packages/shared/pkg/storage/storage_fs.gofsObject.PutfsObject.StoreFilefsObject.getHandle
- Dominant language
- Go
- Stars
- 1.6k
- Forks
- 438
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from e2b-dev/runtime
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
sandbox cache: StartRemoving state transition not broadcast, all allocations see stale Running state Open
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Similar issues
-
textual definition
Difficulty 1/5 Under an hour Newbie friendliness 90/100
geneontology/go-ontology#32653 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
-
needs design
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Priority/High ready-for-agent Severity/Major Type/Bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100