List of a prefix 503s when a single object key contains an empty path segment (//)
维护者通常 1 天内回复
评估
调研方向
首先跟踪 ListObjectsV2 的分页处理，直到 issue 中描述的严格 Path::parse 调用，然后检查现有的列表相关测试。使用包含 // 的键进行复现，并验证单个无效键不再导致整个前缀列表返回 503，同时应跳过有问题的键并发出警告，或以宽松方式处理该键。
由索引模型根据 Issue 内容生成。
描述
Summary
When a ListObjectsV2 page returned by the backend contains a single object whose key has an empty path segment (a //), multistore fails the entire list request with 503 ServiceUnavailable instead of listing the other (valid) keys. One malformed object key takes out the whole prefix.
object_store's strict Path parser rejects empty segments, and multistore propagates that as a fatal backend error for the page rather than skipping the offending key.
Observed error
Proxy log:
[WARN] multistore::proxy: request failed {
error=backend error: Encountered object with invalid path:
Path "raw//" contained empty path segment,
status=503, s3_code=ServiceUnavailable
}
Client just sees the generic:
An error occurred (ServiceUnavailable) when calling the ListObjectsV2 operation
(reached max retries: 2): Service unavailable
Reproduction
An object genuinely exists in S3 with a // in its key (double slash after raw/):
s3://.../raw//b.e21.BW.f09_g17.SSP245-TSMLT-GAUSS-DELAYED-2045.001/atm/proc/tseries/day_1/....nc
aws s3 ls --no-sign-request against the raw bucket lists it fine (S3 keys are opaque byte strings and // is legal). Listing the same prefix through multistore 503s.
Only prefixes that contain such a key fail; sibling prefixes list normally.
Impact
A single stray upload with a double slash (or a zero-byte folder-marker object ending in //) makes an entire prefix unlistable through the gateway, with an error that gives the operator no hint about which object or why. S3 itself accepts and serves these keys, so the gateway is stricter than the backend it fronts.
Suggested fix
On list, don't let one unparseable key fail the page. Options:
- Skip keys that fail strict
Path::parse, ideally logging the raw key atwarnso operators can find and fix it; or - Parse list keys leniently (
Path::from/from_url_path, which tolerate empty segments) instead of strictparse.
At minimum, surface the offending raw key in the error so it's diagnosable without correlating backend logs.
Environment
- multistore 0.6.3 (features:
azure), viamultistore-cf-workerson Cloudflare Workers - Backend: S3
- 主要语言
- Rust
- 星标
- 20
- 派生
- 1
- 平均合并
- 1 天 8 小时
- 30 天内合并 PR
- 10
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
developmentseed/multistore 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 74/100
developmentseed/multistore#160 ·
维护者通常 1 天内回复
-
enhancement
难度 4/5 3-5 天 新手友好度 68/100
developmentseed/multistore#89 ·
维护者通常 1 天内回复
-
oidc-provider: optional `oidc_session_name` backend-option to override RoleSessionName derivation未关闭
难度 2/5 1-3 小时 新手友好度 35/100
developmentseed/multistore#84 ·
维护者通常 1 天内回复
-
Credential cache key (role ARN) doesn't capture the full minted identity (subject + extra_claims)未关闭
难度 4/5 3-5 天 新手友好度 55/100
developmentseed/multistore#81 ·
维护者通常 1 天内回复
-
难度 3/5 1-2 天 新手友好度 65/100
developmentseed/multistore#70 ·
维护者通常 1 天内回复
查看 developmentseed/multistore 的全部 Issue
相似的 Issue
-
bug needs triage P3
难度 2/5 1-3 小时 新手友好度 76/100
pbakaus/impeccable#1143 ·
维护者通常 1 天内回复
-
Bundled uv is never updated and trusts the compromised eMudhra certificates可能已有人在做 @FabienDanieau 今天认领。 未关闭bug
难度 2/5 1-3 小时 新手友好度 76/100
-
难度 2/5 1-3 小时 新手友好度 66/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 66/100
维护者通常 1 天内回复