feat(queue): permanent versus transient consumer errors; undecodable messages to the DLQ; native `dead_letters` table

Aberta
  • Algo errado?
#53 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
5/5
Tempo estimado
Mais de uma semana
Facilidade para iniciantes
28/100
Tipo de issue
Funcionalidade
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
rust, sqlite
Domínio
backend, databases, devops, testing

Direção de pesquisa

Start with the queue handling paths in crates/thrust-worker/src/lib.rs and crates/thrust-adapters-memory/src/lib.rs, then read the EventConsumer contract in crates/thrust-ports/src/lib.rs and the ActivityConsumer mapping in crates/thrust-core/src/lib.rs. Use the listed contract tests and worker probe as the verification entry points. Done means permanent and undecodable messages are recorded once, transient errors retry with backoff, and the ADR, question, and docs reflect the policy.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

area:app prio:p1 review:fable-2026-10-19

Context

The Worker's queue consumer (crates/thrust-worker/src/lib.rs:166-187) decodes each message as an Envelope. Any handler error is retried after 5 s (:172-177); an undecodable body is logged, acked and dropped (:179-183). The port erases the error class: EventConsumer::handle returns Result<(), String> (crates/thrust-ports/src/lib.rs:179-181) and ActivityConsumer turns every CoreError into a string (crates/thrust-core/src/lib.rs:532-536), although ADR-0070's sketch already names a ConsumerError (line 53). The local wrangler.toml sets max_retries = 3 and no dead_letter_queue (crates/thrust-worker/wrangler.toml:39-43), so an exhausted message is deleted; only wrangler.stage2.toml:45-50 names a DLQ. Natively, MemoryQueue discards the consumer's result (crates/thrust-adapters-memory/src/lib.rs:318-322): no retry, no record.

Problem

Spike 0001 Stage 2 (research/0004 §10.2 S2-9, line 371; F4, lines 419-420): an event with no project_id, a permanent CoreError::Invalid (crates/thrust-core/src/lib.rs:434-436), was delivered 4 times, 5 to 6 s apart, then dead-lettered. The undecodable branch could not be driven, because the inject route only accepts a valid Envelope. Q-0006.5 (QUESTIONS.md:358) is open. ADR-0070 asks for retries with backoff and dead-letter handling (line 22; Confirmation "DLQ after N failures", line 73). The docs describe the gap (docs/src/architecture/infra.md:60-63, docs/src/flows/app.md:114-115). M2 adds the notification consumer, and every new consumer copies whatever policy exists.

Proposal

  • thrust-ports: enum ConsumerError { Permanent(String), Transient(String) }, and EventConsumer::handle returns Result<(), ConsumerError>. Core maps Invalid, NotFound and decode failures to Permanent, and DbError::Backend, QueueError and BlobError to Transient.
  • Worker: Permanent: send the body and the error to the DLQ through a producer binding, then ack. Transient: retry_with_options with a delay that grows with msg.attempts(). Undecodable: send the raw text to the DLQ, then ack. wrangler.toml gets the DLQ producer binding and dead_letter_queue on the consumer, as wrangler.stage2.toml:50 has.
  • Native: a dead_letters table (crates/thrust-adapters-postgres/migrations/0002_dead_letters.sql: id, received_at, topic, event_id nullable, body text, error, attempts), written through an untenanted SystemScope method (#35). The in-process queue records permanent failures at once and retries transient ones with backoff before recording them.
  • One view on every target: the Worker consumes its own DLQ into the same table, and a test route GET /internal/dead-letters (test routes on, admin token) reads it, so one contract test runs on all four targets.
  • Fold the policy into ADR-0070 (round-29 note) and answer Q-0006.5 citing FR-21.

Acceptance criteria

  • permanent_consumer_error_is_dead_lettered_without_retry (contract suite, four targets): an Envelope without project_id sent through /internal/inject-event appears once in /internal/dead-letters with attempts = 1. Control: mapping Invalid to Transient makes it fail; add that mutant to scripts/mutants.sh.
  • A thrust-core unit test shows a backend DbError from the memory adapter classified Transient, and a missing project_id classified Permanent.
  • worker_undecodable_message_is_dead_lettered (crates/thrust-contract-tests/tests/worker_only.rs): a queue-raw probe in crates/thrust-worker/src/probes.rs sends a non-JSON body, and the row appears with the raw text. Control: restoring ack-and-drop makes it fail.
  • grep -n "dropping undecodable" crates/thrust-worker/src/lib.rs and grep -n "let _ = c.handle" crates/thrust-adapters-memory/src/lib.rs return nothing.
  • ADR-0070 carries the round-29 note; Q-0006.5 is answered with FR-21; infra.md and flows/app.md describe the new behaviour.

References

  • Review: .agents/plans/contrail/reviews/2026-10-19-fable-architecture-review.md (FR-21)
  • crates/thrust-worker/src/lib.rs:166-187; crates/thrust-ports/src/lib.rs:179-181; crates/thrust-core/src/lib.rs:434-436,532-536; crates/thrust-adapters-memory/src/lib.rs:318-322
  • crates/thrust-worker/wrangler.toml:39-43; crates/thrust-worker/wrangler.stage2.toml:45-50
  • research/0004-rust-on-workers-binding-matrix.md §10.2 (S2-9) and §10.4 (F4)
  • ADR-0070 (lines 22, 53, 73), ADR-0046; Q-0006.5
  • docs/src/architecture/infra.md:60-63; docs/src/flows/app.md:114-115

Blocked by: #43

Finding: FR-21

Linguagem predominante
Rust
Estrelas
0
Forks
0
Merge médio
1h 35min
PRs com merge (30d)
67

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de aRustyDev/thrust

Todas as issues de aRustyDev/thrust

Issues semelhantes

Mais issues de Rust

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.