connectors: define the sink delivery and recovery contract

Abierto
  • ¿Algo no está bien?
#4,233 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
kafka, rust
Área
api, distributed-systems

Línea de trabajo

Start by reading parent tracker #4232, Kafka Connect's SinkTask contract, Iggy's source acknowledgment work in #3855, and the failure scenarios in #4237. Document the selected delivery, checkpoint, retry, lifecycle, filtering, versioning, and migration behavior, including expected outcomes for each scenario. Done means the observable contract and chosen implementation approach are recorded and can be used to judge correctness.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

connectors

Parent tracker: #4232.

Agree on the behavior the runtime and sink plugins must provide before selecting the API shape. The design should make it possible to determine whether an implementation is correct from its observable behavior.

Acceptance criteria:

  • Define the destination acknowledgment required for a record to count as delivered, including buffering, partial success, and uncertain outcomes.
  • Specify the checkpoint rule per partition, including whether offsets identify the last completed record or the next record to process. Define duplicate behavior, the supported ownership model, and source durability and retention assumptions.
  • Choose an acknowledgment mechanism that identifies the records it covers. A serialized flush barrier is acceptable if its completion proves delivery of the entire covered set.
  • Define retry responsibility, failure isolation, shutdown, restart, and assignment changes. Document unsupported concurrency where necessary.
  • Define intentional filtering and the treatment of unexpected pipeline failures.
  • Agree on API versioning, capability checks, defaults, and migration.
  • Record expected outcomes for the failure scenarios used by #4237.

Use Kafka Connect's sink contract and Iggy's existing source acknowledgment work in #3855 as references. The decision should extend the existing discussions and explain which implementation approach is selected.

Lenguaje dominante
Rust
Estrellas
5.2k
Forks
502
Merge medio
3 d 8 h
PR fusionados (30 d)
146

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/iggy

Todos los issues de apache/iggy

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.