Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Propagate the request correlation id to downstream sinks

Aberta
  • Algo errado?
#212 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
3/5
Tempo estimado
1-2 dias
Facilidade para iniciantes
72/100
Tipo de issue
Funcionalidade
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
aws, kafka, python
Domínio
backend

Direção de pesquisa

Leia src/utils/observability.py primeiro para confirmar como os IDs de correlação são resolvidos e, em seguida, rastreie o caminho da chamada em HandlerTopic._write_to_all() até WriterKafka.write() e WriterEventBridge.write(). Implemente a propagação somente por meio de metadados de transporte (header do Kafka ou detail/TraceHeader do EventBridge) e preserve o comportamento quando não houver ID. Execute ou adicione testes unitários com e sem ID de correlação e verificações de integração para os metadados do sink; depois, atualize a seção "Logging & Correlation" do README para documentar o contrato.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

enhancement
Feature Description

Propagate the request correlation id to the downstream sinks, so consumers of the fan-out targets can join their processing logs to the EventGate request that produced the message:

  • Kafka — add the id as a record header (e.g. correlation_id) in WriterKafka.write(). Record headers keep the payload untouched, and consumers can read the header without schema changes.
  • EventBridge — include the id in the put_events entry, either as a top-level field of Detail or via TraceHeader, in WriterEventBridge.write().
  • Postgres — optional; would require a schema change (extra column), so likely out of scope for the first iteration.
Problem / Opportunity

Since #193 / PR #204, every EventGate request resolves a correlation id (X-Correlation-ID → X-Request-ID → API Gateway request id), logs it on every line and returns it in the X-Correlation-ID response header. The id currently stops at EventGate: once a message is fanned out, downstream consumers have no way to correlate their processing with the originating request.

Beneficiaries: teams consuming the Kafka topics and EventBridge events, and anyone debugging a cross-system flow end-to-end — one id would then trace a message from the caller, through EventGate's logs, into every sink's consumer logs.

Acceptance Criteria
  1. A message published to Kafka carries the request's correlation id as a record header; the message payload (value) is byte-for-byte unchanged.
  2. An event published to EventBridge carries the request's correlation id in its entry metadata (TraceHeader or a documented Detail field); the schema-validated payload is unchanged.
  3. When no correlation id is resolved (empty id), no header/field is added and writes behave exactly as today.
  4. Existing topic schema validation and all existing consumers are unaffected — propagation uses transport metadata only, never the message body.
  5. Unit tests cover both writers with and without a bound correlation id; integration tests prove the header/field arrives in the sink.
  6. README "Logging & Correlation" documents the downstream propagation contract.
Proposed Solution

The writers do not currently receive the correlation id; it is bound in src/utils/observability.py. Two options:

  • a small accessor (e.g. current_correlation_id()) in src/utils/observability.py, read by the writers, or
  • an explicit parameter threaded through HandlerTopic._write_to_all() into Writer.write().

The explicit parameter is more testable and keeps writers free of hidden state; the accessor avoids touching the Writer interface. Alternative considered and rejected: putting the id into the message body — breaks topic schema validation and changes consumer contracts.

The id is already validated against ^[A-Za-z0-9._:-]{1,128}$ before use, so it is safe to forward verbatim.

Dependencies / Related
  • #193 (structured logging + correlation id)
  • PR #204 (implementation; ADR adr/001-observability/001-observability.md)
Additional Context

Follow-up proposed during the second-opinion review of PR #204 (item P7). Postgres propagation can be a separate issue if a schema change is ever justified.

Linguagem predominante
Python
Estrelas
4
Forks
0
Merge médio
22h 3min
PRs com merge (30d)
8

Preparar o ambiente

  • Inclui um Dockerfile ou arquivo Docker Compose
  • Tem um modelo de pull request
  • Sem guia de contribuição

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de AbsaOSS/EventGate

Todas as issues de AbsaOSS/EventGate

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.