Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Stale activity cleanup: TTL or background sweep for undeliverable worker queue items

Open
  • Is something wrong?
#3 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
postgresql, rust
Domain
backend, databases, distributed-systems

Research direction

Start with the TODO.md entry and trace the provider trait, worker queue items, RuntimeOptions, and activity builder described in the issue, including the CosmosDB and Postgres providers. Decide which expiry design and retry semantics fit the existing error infrastructure, then verify that expired items are failed back to the orchestrator while the default remains no expiry.

Written by the indexing model from the issue text.

Description

enhancement

Problem

Activities scheduled with a tag that no worker is configured to handle (or activities whose target worker goes offline permanently) will sit in the worker queue indefinitely. The orchestration that scheduled them hangs forever unless the user manually implements a select2(activity, timer) starvation guard.

This is especially relevant now that activity tagging is implemented -- it is easy to schedule a .with_tag("gpu") activity in an environment where no GPU worker is running.

Desired Behavior

Undeliverable or stale activities should not block orchestrations forever. The runtime should detect activities that exceed a configurable time limit and fail them back to the orchestrator with a clear error.

Proposed Approaches

Option A: Activity TTL (per-item expiry)
  • Add an optional expires_at timestamp to worker queue items (set at enqueue time based on a configurable TTL)
  • Provider fetch_work_item() skips expired items
  • A periodic sweep (or check at fetch time) marks expired items as failed
  • The orchestration receives an ActivityExpired error it can match on
Option B: Background cleanup process
  • A runtime background task periodically scans for worker queue items older than a configurable threshold
  • Stale items are failed back to the orchestrator with a timeout error
  • Simpler to implement but less granular (global threshold vs per-activity)
Option C: Hybrid
  • Default global TTL from RuntimeOptions (e.g., 1 hour)
  • Per-activity override via .with_ttl(Duration) on the activity builder
  • Background sweep handles the cleanup

Design Considerations

  • Provider trait changes: Need expires_at field or equivalent on worker queue items
  • Event model: New ActivityExpired or reuse existing error infrastructure
  • Backward compatibility: TTL should be optional, default to no expiry (current behavior) for existing users
  • CosmosDB / Postgres providers: Both need the expiry field; CosmosDB has native TTL support that could be leveraged
  • Interaction with retries: Should TTL apply per-attempt or total? Probably total elapsed since first enqueue

Related

  • Activity tagging feature (.with_tag() / TagFilter)
  • select2(activity, timer) starvation-safe pattern (current workaround)
  • TODO.md entry added for tracking
Dominant language
Rust
Stars
217
Forks
61
Avg merge
3d 9m
Merged PRs (30d)
2

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/duroxide

All issues in microsoft/duroxide

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.