S3 UploadPartCopy failures leak reserved upload size
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 3/5
- Estimated time
- Half a day
- Newbie friendliness
- 42/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- postgresql, typescript
- Domain
- backend
Research direction
Start with the UploadPartCopy handler and shouldAllowPartUpload(), where the copy size is reserved in storage.s3_multipart_uploads.in_progress_size before the backend call and part-row insert. Compare against the existing UploadPart error path that already releases its reservation, and add the same release for backend and insert failures. Done when a retry of a failed copy under the 9-byte limit succeeds, and the reproduction steps in the issue pass.
Written by the indexing model from the issue text.
Description
Bug report
A failed UploadPartCopy leaves its bytes in storage.s3_multipart_uploads.in_progress_size. A valid retry can then fail with 413 EntityTooLarge before reaching the recovered backend.
The handler reserves the copy size through shouldAllowPartUpload(), then calls the backend and inserts the part row without an error cleanup path. Ordinary UploadPart failures already release their reservation.
To reproduce
- Create a bucket with a 9-byte file size limit and upload a 9-byte source object.
- Start a multipart upload for a different object.
- Make the backend fail one
UploadPartCopyrequest, for example with a one-shot injected backend error. - Retry the same copy after the backend recovers.
A local AWS SDK run against PostgreSQL 15 and RustFS, at upstream b74897cf, produced:
{"step":"after-failed-copy","progress":9}
EntityTooLarge: The object exceeded the maximum allowed size
The first request stored no part. Its retry still fails because the handler counts the failed copy's 9-byte reservation.
Expected behavior
Release the failed request's reservation while preserving bytes from other successful or active parts. The retry should fit the original limit and complete normally. A failed part-row insert after the backend copy needs the same cleanup.
Scope
This concerns failures after a copy reservation. It is separate from #1493/#1494, which checks source preconditions before reservation, and #1436/#1437, which covers duplicate part listings. Successful part replacement accounting and HTTP disconnect cleanup are outside this report.
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 318
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 104
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from supabase/storage
-
S3 CreateBucket returns BucketAlreadyExists for a bucket the project already ownsPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 1 day
-
The sample application in storage article points to wrong URLMay be free again @prasadf18 claimed this 208 days ago, and no pull request is open. Opendocumentation
Difficulty 1/5 Under an hour Newbie friendliness 62/100
supabase/storage#557 · 1 comment ·
Maintainers usually reply within 1 day
-
REST downloads from the S3 backend drop ETag and Cache-Control on 304 responsesPossibly taken @hsusul claimed this 1 day ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
Local image transformations fail for filenames containing @Possibly taken @hsusul claimed this 3 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 62/100
Maintainers usually reply within 1 day
All issues in supabase/storage
Similar issues
-
needs-area-label
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
liberation-data/drivine#615 ·
Maintainers usually reply within 1 day
-
backend bug ready-for-agent
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
geolens-io/geolens#3639 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
radius-project/ai-extensions#1196 ·
Maintainers usually reply within 1 day
-
good first issue
Difficulty 1/5 1-3 hours Newbie friendliness 88/100