datalake_agent: Iceberg operations by metadata location
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 35/100
调研方向
Start with the B2 conformance suite and the B1 dependency, then trace the iceberg-java integration and ResolvingFileIO entry points, including S3FileIO. Use MinIO for end-to-end checks. Done means the listed create, load, plan, and stage operations pass, retries create one metadata file, logs omit credentials, and memory remains flat over ten thousand plan calls.
由索引模型根据 Issue 内容生成。
描述
Part of #2008. Letters (A, B0–B7, C, D, E) are the PRs listed there; this is B3.
Scope
The kernel behind B1, with no catalog service yet: every operation takes a metadata_location. This is what the Builtin catalog (B4) needs and what every catalog provider (B5–B7) builds on.
- Through iceberg-java: initial metadata for a new table at a location; load, statistics, the schema changes the AM allows, truncate.
- Planning: a snapshot's data files, delete files and row-group ranges streamed as fragments; file groups for rewrite.
- Staging: append and update per statement, producing a new uncommitted metadata file whose location the caller commits (B4 locally, B5–B7 through the catalog).
- An
operation_idclaim store so a retried request applies once. - Storage from the request:
ResolvingFileIOwithS3FileIOfors3://. - iceberg-java moves from the 1.3 line the port started on to a current 1.x.
- In-process unit tests per RPC; end-to-end against MinIO alone.
Out of scope
Catalog providers (B5–B7); compaction and snapshot expiry; data-file I/O (the agent never touches data files).
Depends on
B1; verified through B2.
Acceptance
- Create, load, plan and stage against MinIO alone, driven by the B2 conformance suite.
- A staged append retried with the same
operation_idyields one metadata file. - Credentials never appear in agent logs; agent memory is flat over ten thousand plan calls.
- 主要语言
- C
- 星标
- 1.4k
- 派生
- 248
- 平均合并
- 4 天 10 小时
- 30 天内合并 PR
- 40
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
apache/cloudberry 的其他 Issue
-
type: Bug
难度 2/5 1-3 小时 新手友好度 76/100
apache/cloudberry#1885 · 2 个 reaction ·
-
难度 2/5 1-3 小时 新手友好度 86/100
apache/cloudberry#1825 ·
-
type: Bug
难度 3/5 1-2 天 新手友好度 65/100
apache/cloudberry#2048 · 1 个 reaction ·
-
type: Bug
难度 4/5 3-5 天 新手友好度 40/100
apache/cloudberry#2047 ·
-
type: Bug
难度 4/5 3-5 天 新手友好度 45/100
apache/cloudberry#2046 · 1 条评论 ·
查看 apache/cloudberry 的全部 Issue
相似的 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 75/100
bradcypert/plum#53 ·
-
Component: GLib
难度 2/5 1-3 小时 新手友好度 70/100
-
难度 2/5 1-3 小时 新手友好度 75/100
-
Status: Opened
难度 2/5 1-3 小时 新手友好度 70/100
-
难度 2/5 1-3 小时 新手友好度 75/100
nextbsd/nextbsd-userland#285 ·