feat(queue): permanent versus transient consumer errors; undecodable messages to the DLQ; native `dead_letters` table

Abierto
  • ¿Algo no está bien?
#53 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
28/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
rust, sqlite
Área
backend, databases, devops, testing

Línea de trabajo

Start with the queue handling paths in crates/thrust-worker/src/lib.rs and crates/thrust-adapters-memory/src/lib.rs, then read the EventConsumer contract in crates/thrust-ports/src/lib.rs and the ActivityConsumer mapping in crates/thrust-core/src/lib.rs. Use the listed contract tests and worker probe as the verification entry points. Done means permanent and undecodable messages are recorded once, transient errors retry with backoff, and the ADR, question, and docs reflect the policy.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

area:app prio:p1 review:fable-2026-10-19

Context

The Worker's queue consumer (crates/thrust-worker/src/lib.rs:166-187) decodes each message as an Envelope. Any handler error is retried after 5 s (:172-177); an undecodable body is logged, acked and dropped (:179-183). The port erases the error class: EventConsumer::handle returns Result<(), String> (crates/thrust-ports/src/lib.rs:179-181) and ActivityConsumer turns every CoreError into a string (crates/thrust-core/src/lib.rs:532-536), although ADR-0070's sketch already names a ConsumerError (line 53). The local wrangler.toml sets max_retries = 3 and no dead_letter_queue (crates/thrust-worker/wrangler.toml:39-43), so an exhausted message is deleted; only wrangler.stage2.toml:45-50 names a DLQ. Natively, MemoryQueue discards the consumer's result (crates/thrust-adapters-memory/src/lib.rs:318-322): no retry, no record.

Problem

Spike 0001 Stage 2 (research/0004 §10.2 S2-9, line 371; F4, lines 419-420): an event with no project_id, a permanent CoreError::Invalid (crates/thrust-core/src/lib.rs:434-436), was delivered 4 times, 5 to 6 s apart, then dead-lettered. The undecodable branch could not be driven, because the inject route only accepts a valid Envelope. Q-0006.5 (QUESTIONS.md:358) is open. ADR-0070 asks for retries with backoff and dead-letter handling (line 22; Confirmation "DLQ after N failures", line 73). The docs describe the gap (docs/src/architecture/infra.md:60-63, docs/src/flows/app.md:114-115). M2 adds the notification consumer, and every new consumer copies whatever policy exists.

Proposal

  • thrust-ports: enum ConsumerError { Permanent(String), Transient(String) }, and EventConsumer::handle returns Result<(), ConsumerError>. Core maps Invalid, NotFound and decode failures to Permanent, and DbError::Backend, QueueError and BlobError to Transient.
  • Worker: Permanent: send the body and the error to the DLQ through a producer binding, then ack. Transient: retry_with_options with a delay that grows with msg.attempts(). Undecodable: send the raw text to the DLQ, then ack. wrangler.toml gets the DLQ producer binding and dead_letter_queue on the consumer, as wrangler.stage2.toml:50 has.
  • Native: a dead_letters table (crates/thrust-adapters-postgres/migrations/0002_dead_letters.sql: id, received_at, topic, event_id nullable, body text, error, attempts), written through an untenanted SystemScope method (#35). The in-process queue records permanent failures at once and retries transient ones with backoff before recording them.
  • One view on every target: the Worker consumes its own DLQ into the same table, and a test route GET /internal/dead-letters (test routes on, admin token) reads it, so one contract test runs on all four targets.
  • Fold the policy into ADR-0070 (round-29 note) and answer Q-0006.5 citing FR-21.

Acceptance criteria

  • permanent_consumer_error_is_dead_lettered_without_retry (contract suite, four targets): an Envelope without project_id sent through /internal/inject-event appears once in /internal/dead-letters with attempts = 1. Control: mapping Invalid to Transient makes it fail; add that mutant to scripts/mutants.sh.
  • A thrust-core unit test shows a backend DbError from the memory adapter classified Transient, and a missing project_id classified Permanent.
  • worker_undecodable_message_is_dead_lettered (crates/thrust-contract-tests/tests/worker_only.rs): a queue-raw probe in crates/thrust-worker/src/probes.rs sends a non-JSON body, and the row appears with the raw text. Control: restoring ack-and-drop makes it fail.
  • grep -n "dropping undecodable" crates/thrust-worker/src/lib.rs and grep -n "let _ = c.handle" crates/thrust-adapters-memory/src/lib.rs return nothing.
  • ADR-0070 carries the round-29 note; Q-0006.5 is answered with FR-21; infra.md and flows/app.md describe the new behaviour.

References

  • Review: .agents/plans/contrail/reviews/2026-10-19-fable-architecture-review.md (FR-21)
  • crates/thrust-worker/src/lib.rs:166-187; crates/thrust-ports/src/lib.rs:179-181; crates/thrust-core/src/lib.rs:434-436,532-536; crates/thrust-adapters-memory/src/lib.rs:318-322
  • crates/thrust-worker/wrangler.toml:39-43; crates/thrust-worker/wrangler.stage2.toml:45-50
  • research/0004-rust-on-workers-binding-matrix.md §10.2 (S2-9) and §10.4 (F4)
  • ADR-0070 (lines 22, 53, 73), ADR-0046; Q-0006.5
  • docs/src/architecture/infra.md:60-63; docs/src/flows/app.md:114-115

Blocked by: #43

Finding: FR-21

Lenguaje dominante
Rust
Estrellas
0
Forks
0
Merge medio
1 h 35 min
PR fusionados (30 d)
67

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de aRustyDev/thrust

Todos los issues de aRustyDev/thrust

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.