feat(jobs): daily hard-delete retention cleanup (F-D10, 30-day default per D-001)

Aberta
#59 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
5/5
Tempo estimado
Mais de uma semana
Facilidade para iniciantes
25/100
Tipo de issue
Funcionalidade
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
postgresql, rust
Domínio
backend, database, devops

Direção de pesquisa

Start with CoreConfig and the soft delete and blob store interfaces in crates/thrust-core/src/lib.rs and crates/thrust-ports/src/lib.rs; then read the worker cron handler, native server timer, and schedules.rs. The issue is blocked by #35 and #43, so those dependencies need to land first. Done means retention deletes only rows past the configured cutoff, removes their blobs and eligible outbox rows, and passes the specified core, adapter, route, and scheduling tests.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

area:app prio:p0 review:fable-2026-10-19

Context

Only work items are soft-deleted today. delete_work_item (crates/thrust-core/src/lib.rs:301-344) reads the injected Clock (:309) and sets work_items.deleted_at (crates/thrust-adapters-postgres/migrations/0001_init.sql:40) through Tx::soft_delete_work_item (crates/thrust-ports/src/lib.rs:147-153; Postgres at crates/thrust-adapters-postgres/src/lib.rs:444-455). Nothing ever removes a row: a search of crates/, scripts/ and docs/src/ for hard_delete, purge or retention finds only the R2 binding probe, and the outbox table (0001_init.sql:54-62) only grows. The only scheduled work is the outbox sweeper. The Worker's single cron, * * * * * (crates/thrust-worker/wrangler.toml:20-21), runs it on every trigger; the handler only logs event.cron() and does not branch on it (crates/thrust-worker/src/lib.rs:193-213). The native server runs it on an optional interval (crates/thrust-server/src/main.rs:93-106). ADR-0072's SCHEDULES table in crates/thrust-core/src/schedules.rs (ADR-0072 lines 38-46) does not exist.

Problem

  • The plan requires this cleanup. F-D10 is P0 with hard delete at 30 days (FEATURES.md:83). D-001 records the 30-day default against Plane's 60 (DEVIATIONS.md:9). Q-0043.2 keeps the daily cadence (QUESTIONS.md:181). ADR-0043 notes that nothing else would purge soft-deleted rows (line 93). The review puts the cleanup in M2 (§3, FR-01), and TRACEABILITY asks for a cron test and a retention test (TRACEABILITY.md:54).
  • Plane at the pin: hard_delete runs daily at 00:00 UTC (apps/api/plane/celery.py:61-63) and reads HARD_DELETE_AFTER_DAYS (settings/common.py:420, default 60). It deletes rows with deleted_at < now - days, first from 18 named models (bgtasks/deletion_task.py:138-180) and then from every model that has a deleted_at field (:182-191). The comparison is strict, so a row deleted exactly N days ago stays.
  • ADR-0046 keeps outbox rows for 90 days and says the retention job is not built (lines 32, 71, 120). Under FR-18 the retained outbox is the P0 audit trail, so this purge also sets the P0 audit retention (Q-0048.1).
  • The purge runs across tenants, so it cannot take a Scope. Attachments have no row: they are blobs under <workspace>/<project>/<item>/ (crates/thrust-core/src/lib.rs:468-470), so purging an item's row would leave its blobs behind.

Proposal

  • Config: CoreConfig (crates/thrust-core/src/lib.rs:29-47) gains hard_delete_after_days: u32 (default 30, D-001) and outbox_retention_days: u32 (default 90), read from THRUST_HARD_DELETE_AFTER_DAYS and THRUST_OUTBOX_RETENTION_DAYS. A negative or unparseable value fails startup, for the reason Plane gives at settings/common.py:423-436.
  • thrust_core::run_retention(app, &SystemScope) takes the SystemScope from #35. With cutoff = clock.now() - days, it deletes rows with deleted_at < cutoff in bounded batches, children before parents, from every soft-deletable table (work_items today; later tables join a registry). For each purged item it also deletes the blobs under the item's prefix (BlobStore::list and delete, crates/thrust-ports/src/lib.rs:200-205). Then it deletes outbox rows that were created more than 90 days ago and have been dispatched (created_at < now - 90 d AND dispatched_at IS NOT NULL); it never purges an undispatched row.
  • What the purge leaves: project_sequences is never touched, so a purged number is never handed out again (ADR-0043 B1). Activity rows are kept, as Plane declares IssueActivity.issue with on_delete=DO_NOTHING (apps/api/plane/db/models/issue.py:416).
  • Scheduling: schedules.rs holds two entries, outbox-sweep (* * * * *) and hard-delete (0 0 * * *).
    • The single Worker from #43 adds "0 0 * * *" to triggers.crons and dispatches on event.cron().
    • The native server adds a daily timer at 00:00 UTC. It runs as a single process (ADR-0070 round 29), so no leader election is needed (ADR-0072 line 56).
    • Each run works through bounded batches inline, within the 15-minute cron wall time (FR-05), instead of enqueuing work as ADR-0072 line 22 says; record that in ADR-0072. A run is idempotent.
  • Test route: POST /internal/retention takes a now override. Like /internal/sweep (crates/thrust-api/src/lib.rs:158-160,275), it needs test routes and the admin token, so the contract suite can drive it on all four targets.
  • F-D10's log cleanups (API, email and webhook logs, versions) join this job once their tables exist.

Acceptance criteria

  • crates/thrust-core/tests/retention.rs passes. It is hermetic: memory adapters, with a settable clock in place of FixedClock (tests/common/mod.rs:33-39). With 30-day retention, in both tenants, items soft-deleted 31 days and 30 days plus 1 s ago are gone; items deleted exactly 30 days ago and 29 days ago, and live items, remain; a later create gets the next sequence number. Control: two mutants added to scripts/mutants.sh, one comparing with <= and one ignoring the cutoff, each make it fail.
  • The outbox follows the same rule: a dispatched row created 91 days ago is gone, while a dispatched row created 89 days ago and an undispatched row created 200 days ago remain. Control: a mutant that purges on age alone makes it fail.
  • A purged item's prefix lists no blobs, and another item's blobs remain.
  • retention_purges_only_rows_past_the_window passes through /internal/retention on all four targets. pg_adapter.rs covers the purge SQL on real Postgres, with the cutoff bound as a typed parameter (execute_typed).
  • In worker_only.rs, next to worker_cron_scheduled_handler_runs_the_sweeper (:110-134), with THRUST_HARD_DELETE_AFTER_DAYS=0: after /__scheduled?cron=*+*+*+*+*, a deleted item is still listed with scope=with_deleted; after /__scheduled?cron=0+0+*+*+*, it is gone. Control: a handler that ignores event.cron() makes it fail.
  • A test asserts that wrangler.toml triggers.crons and SCHEDULES hold the same cron set. Adding a cron to only one of them makes it fail.

References

  • Review: .agents/plans/contrail/reviews/2026-10-19-fable-architecture-review.md (FR-01; §3 M2 row; FR-18 (3))
  • FEATURES F-D10 (FEATURES.md:83); DEVIATIONS D-001 (DEVIATIONS.md:9); TRACEABILITY F-D10 (TRACEABILITY.md:54); Q-0043.2 (QUESTIONS.md:181), Q-0046.2 (:185), Q-0073.2 (:110), Q-0048.1 (:279)
  • ADR-0043 (lines 79, 89, 93), ADR-0072 (lines 22, 38-46, 56, 60-61, 89), ADR-0046 (lines 32, 71, 120), ADR-0070 (round-29 note), ADR-0045
  • crates/thrust-core/src/lib.rs:29-47,301-344,468-470; crates/thrust-ports/src/lib.rs:147-153,200-205; crates/thrust-adapters-postgres/src/lib.rs:108-127,444-455; crates/thrust-adapters-postgres/migrations/0001_init.sql:40,54-62
  • crates/thrust-worker/wrangler.toml:20-21; crates/thrust-worker/src/lib.rs:193-213; crates/thrust-server/src/main.rs:93-106; crates/thrust-api/src/lib.rs:158-160,275; crates/thrust-core/tests/common/mod.rs:33-39; scripts/mutants.sh:30-32
  • Plane oracle (oracle/v1-pin-20261018): apps/api/plane/bgtasks/deletion_task.py:114-193; apps/api/plane/celery.py:61-63; apps/api/plane/settings/common.py:420,423-448; apps/api/plane/db/models/issue.py:416

Blocked by: #35, #43

Finding: FR-01

Linguagem predominante
Rust
Estrelas
0
Forks
0
Merge médio
1h 35min
PRs com merge (30d)
67

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de aRustyDev/thrust

Todas as issues de aRustyDev/thrust

Issues semelhantes

Mais issues de Rust

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.