feat(queue): permanent versus transient consumer errors; undecodable messages to the DLQ; native `dead_letters` table

Offen
  • Stimmt etwas nicht?
#53 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
28/100
Issue-Typ
Feature
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
rust, sqlite
Bereich
backend, databases, devops, testing

Rechercherichtung

Start with the queue handling paths in crates/thrust-worker/src/lib.rs and crates/thrust-adapters-memory/src/lib.rs, then read the EventConsumer contract in crates/thrust-ports/src/lib.rs and the ActivityConsumer mapping in crates/thrust-core/src/lib.rs. Use the listed contract tests and worker probe as the verification entry points. Done means permanent and undecodable messages are recorded once, transient errors retry with backoff, and the ADR, question, and docs reflect the policy.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

area:app prio:p1 review:fable-2026-10-19

Context

The Worker's queue consumer (crates/thrust-worker/src/lib.rs:166-187) decodes each message as an Envelope. Any handler error is retried after 5 s (:172-177); an undecodable body is logged, acked and dropped (:179-183). The port erases the error class: EventConsumer::handle returns Result<(), String> (crates/thrust-ports/src/lib.rs:179-181) and ActivityConsumer turns every CoreError into a string (crates/thrust-core/src/lib.rs:532-536), although ADR-0070's sketch already names a ConsumerError (line 53). The local wrangler.toml sets max_retries = 3 and no dead_letter_queue (crates/thrust-worker/wrangler.toml:39-43), so an exhausted message is deleted; only wrangler.stage2.toml:45-50 names a DLQ. Natively, MemoryQueue discards the consumer's result (crates/thrust-adapters-memory/src/lib.rs:318-322): no retry, no record.

Problem

Spike 0001 Stage 2 (research/0004 §10.2 S2-9, line 371; F4, lines 419-420): an event with no project_id, a permanent CoreError::Invalid (crates/thrust-core/src/lib.rs:434-436), was delivered 4 times, 5 to 6 s apart, then dead-lettered. The undecodable branch could not be driven, because the inject route only accepts a valid Envelope. Q-0006.5 (QUESTIONS.md:358) is open. ADR-0070 asks for retries with backoff and dead-letter handling (line 22; Confirmation "DLQ after N failures", line 73). The docs describe the gap (docs/src/architecture/infra.md:60-63, docs/src/flows/app.md:114-115). M2 adds the notification consumer, and every new consumer copies whatever policy exists.

Proposal

  • thrust-ports: enum ConsumerError { Permanent(String), Transient(String) }, and EventConsumer::handle returns Result<(), ConsumerError>. Core maps Invalid, NotFound and decode failures to Permanent, and DbError::Backend, QueueError and BlobError to Transient.
  • Worker: Permanent: send the body and the error to the DLQ through a producer binding, then ack. Transient: retry_with_options with a delay that grows with msg.attempts(). Undecodable: send the raw text to the DLQ, then ack. wrangler.toml gets the DLQ producer binding and dead_letter_queue on the consumer, as wrangler.stage2.toml:50 has.
  • Native: a dead_letters table (crates/thrust-adapters-postgres/migrations/0002_dead_letters.sql: id, received_at, topic, event_id nullable, body text, error, attempts), written through an untenanted SystemScope method (#35). The in-process queue records permanent failures at once and retries transient ones with backoff before recording them.
  • One view on every target: the Worker consumes its own DLQ into the same table, and a test route GET /internal/dead-letters (test routes on, admin token) reads it, so one contract test runs on all four targets.
  • Fold the policy into ADR-0070 (round-29 note) and answer Q-0006.5 citing FR-21.

Acceptance criteria

  • permanent_consumer_error_is_dead_lettered_without_retry (contract suite, four targets): an Envelope without project_id sent through /internal/inject-event appears once in /internal/dead-letters with attempts = 1. Control: mapping Invalid to Transient makes it fail; add that mutant to scripts/mutants.sh.
  • A thrust-core unit test shows a backend DbError from the memory adapter classified Transient, and a missing project_id classified Permanent.
  • worker_undecodable_message_is_dead_lettered (crates/thrust-contract-tests/tests/worker_only.rs): a queue-raw probe in crates/thrust-worker/src/probes.rs sends a non-JSON body, and the row appears with the raw text. Control: restoring ack-and-drop makes it fail.
  • grep -n "dropping undecodable" crates/thrust-worker/src/lib.rs and grep -n "let _ = c.handle" crates/thrust-adapters-memory/src/lib.rs return nothing.
  • ADR-0070 carries the round-29 note; Q-0006.5 is answered with FR-21; infra.md and flows/app.md describe the new behaviour.

References

  • Review: .agents/plans/contrail/reviews/2026-10-19-fable-architecture-review.md (FR-21)
  • crates/thrust-worker/src/lib.rs:166-187; crates/thrust-ports/src/lib.rs:179-181; crates/thrust-core/src/lib.rs:434-436,532-536; crates/thrust-adapters-memory/src/lib.rs:318-322
  • crates/thrust-worker/wrangler.toml:39-43; crates/thrust-worker/wrangler.stage2.toml:45-50
  • research/0004-rust-on-workers-binding-matrix.md §10.2 (S2-9) and §10.4 (F4)
  • ADR-0070 (lines 22, 53, 73), ADR-0046; Q-0006.5
  • docs/src/architecture/infra.md:60-63; docs/src/flows/app.md:114-115

Blocked by: #43

Finding: FR-21

Vorherrschende Sprache
Rust
Sterne
0
Forks
0
Ø Merge
1 Std. 35 Min.
Gemergte PRs (30 T.)
67

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus aRustyDev/thrust

Alle Issues in aRustyDev/thrust

Ähnliche Issues

Weitere Issues zu Rust

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.