Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

datalake_agent: Iceberg operations by metadata location

Abierto
#2,013 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
aws, java
Área
backend, cloud, data-engineering, databases, testing

Línea de trabajo

Start with the B2 conformance suite and the B1 dependency, then trace the iceberg-java integration and ResolvingFileIO entry points, including S3FileIO. Use MinIO for end-to-end checks. Done means the listed create, load, plan, and stage operations pass, retries create one metadata file, logs omit credentials, and memory remains flat over ten thousand plan calls.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

datalake

Part of #2008. Letters (A, B0–B7, C, D, E) are the PRs listed there; this is B3.

Scope

The kernel behind B1, with no catalog service yet: every operation takes a metadata_location. This is what the Builtin catalog (B4) needs and what every catalog provider (B5–B7) builds on.

  • Through iceberg-java: initial metadata for a new table at a location; load, statistics, the schema changes the AM allows, truncate.
  • Planning: a snapshot's data files, delete files and row-group ranges streamed as fragments; file groups for rewrite.
  • Staging: append and update per statement, producing a new uncommitted metadata file whose location the caller commits (B4 locally, B5–B7 through the catalog).
  • An operation_id claim store so a retried request applies once.
  • Storage from the request: ResolvingFileIO with S3FileIO for s3://.
  • iceberg-java moves from the 1.3 line the port started on to a current 1.x.
  • In-process unit tests per RPC; end-to-end against MinIO alone.
Out of scope

Catalog providers (B5–B7); compaction and snapshot expiry; data-file I/O (the agent never touches data files).

Depends on

B1; verified through B2.

Acceptance
  • Create, load, plan and stage against MinIO alone, driven by the B2 conformance suite.
  • A staged append retried with the same operation_id yields one metadata file.
  • Credentials never appear in agent logs; agent memory is flat over ten thousand plan calls.
Lenguaje dominante
C
Estrellas
1.4k
Forks
248
Merge medio
4 d 10 h
PR fusionados (30 d)
40

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/cloudberry

Todos los issues de apache/cloudberry

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.