Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Make Writes Idempotent

Abierto
  • ¿Algo no está bien?
#234 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
32/100
Tipo de issue
Nueva funcionalidad
Claridad
Necesita aclaración
Estado de actividad
Activo
Stack tecnológico
kafka, python
Área
backend, databases, distributed-systems

Línea de trabajo

Start by tracing the EventGate Lambda path that performs the DB, EventBridge, and Kafka writes, then inspect the existing DB tables and processing-start data. Reproduce or reason through message redelivery after one write fails. Done means redelivery is handled idempotently, arrival timing is documented or captured as needed, and tests or manual evidence cover the scenario.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement
Feature Description

When the same event arrives multiple times, there is no guarantee that it will not be re-inserted into the DB, or re-delivered to our consumers multiple times. There is no idempotency guarantee.

Problem / Opportunity

The EventGate Lambda handling events and performing writes, which is the central to the EventGate system, currently perform triple write: DB, EventBridge, and Kafka. If there is a failure, e.g. Kafka write fails, the Lambda finishes (the two writes are done) but it constructs response payload with HTTP status 500.

Then, the producer is notified about 500 and maybe it stores the msg into DLQ or redelivers the message again immediatelly, doesn't matter - the message is processed again and the triple write happens again - even if it previously succeeded for some of the writes. Therefore, we'll have duplicit items in our DB and our EventBridge/Kafka consumers might receive the message multiple times potentially.

Something like this happened with Unify recently (Unify as consumer), although no producer complained about this. But this certainly is something we should deal with and redesign.

Note: DB tables contain almost no constraints and very little auxiliary timestamp-based columns - like when a message arrives, a new column arrivedAt with default NOW() should be present - an idea - because otherwise we DO NOT KNOW when the message actually arrived (well, in this case, when it was stored in DB, the arrival happens seconds earlier). The processing start, column that is present, is set by producer, not by eventgate system. On producer's retries, this is the same value!

Acceptance Criteria
  • Scenario with msg-redelivery is handled in better and more expected way, idempotency based ideally.
  • Documentation
  • Either programmatic tests exist, or the final solution was tested at least manually and evidence is attached somewhere, maybe into this ticket even.
Lenguaje dominante
Python
Estrellas
4
Forks
0
Merge medio
3 d 7 h
PR fusionados (30 d)
11

Preparar el entorno

  • Incluye un Dockerfile o un archivo de Docker Compose
  • Tiene una plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de AbsaOSS/EventGate

Todos los issues de AbsaOSS/EventGate

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.