Hacktoberfest 2026:维护者为十月标记出来的 issue,仍然开放、适合新手。 浏览 Hacktoberfest issue

datalake_agent: Iceberg operations by metadata location

未关闭
#2,013 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
aws, java

调研方向

Start with the B2 conformance suite and the B1 dependency, then trace the iceberg-java integration and ResolvingFileIO entry points, including S3FileIO. Use MinIO for end-to-end checks. Done means the listed create, load, plan, and stage operations pass, retries create one metadata file, logs omit credentials, and memory remains flat over ten thousand plan calls.

由索引模型根据 Issue 内容生成。

描述

datalake

Part of #2008. Letters (A, B0–B7, C, D, E) are the PRs listed there; this is B3.

Scope

The kernel behind B1, with no catalog service yet: every operation takes a metadata_location. This is what the Builtin catalog (B4) needs and what every catalog provider (B5–B7) builds on.

  • Through iceberg-java: initial metadata for a new table at a location; load, statistics, the schema changes the AM allows, truncate.
  • Planning: a snapshot's data files, delete files and row-group ranges streamed as fragments; file groups for rewrite.
  • Staging: append and update per statement, producing a new uncommitted metadata file whose location the caller commits (B4 locally, B5–B7 through the catalog).
  • An operation_id claim store so a retried request applies once.
  • Storage from the request: ResolvingFileIO with S3FileIO for s3://.
  • iceberg-java moves from the 1.3 line the port started on to a current 1.x.
  • In-process unit tests per RPC; end-to-end against MinIO alone.
Out of scope

Catalog providers (B5–B7); compaction and snapshot expiry; data-file I/O (the agent never touches data files).

Depends on

B1; verified through B2.

Acceptance
  • Create, load, plan and stage against MinIO alone, driven by the B2 conformance suite.
  • A staged append retried with the same operation_id yields one metadata file.
  • Credentials never appear in agent logs; agent memory is flat over ten thousand plan calls.
主要语言
C
星标
1.4k
派生
248
平均合并
4 天 10 小时
30 天内合并 PR
40

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

apache/cloudberry 的其他 Issue

查看 apache/cloudberry 的全部 Issue

相似的 Issue

更多 C Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。