Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

worker_reliability_test: crash tests fail at random and never test a crash before the completion is saved

オープン
  • 何か問題がありますか？
#69 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
74/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
rust
領域
testing

調査の方向性

Read the two tests in tests/worker_reliability_test.rs, then inspect tests/common/mod.rs and the worker shutdown and lock renewal paths named in the issue. Replace the racy history polling with a signal that confirms each activity has started, and set lock timing through RuntimeOptions. Done when both tests reliably exercise redelivery after shutdown, complete successfully, and confirm the activities started twice.

索引モデルが issue の本文から書いたものです。

説明

bug

Summary

Two tests in tests/worker_reliability_test.rs have the same three problems:

  • activity_reliability_after_crash_before_completion_enqueue
  • multiple_activities_reliability_after_crash
  1. They fail at random.
  2. They never test the case their names describe.
  3. They set an environment variable that nothing reads.

The first test failed once in a local ./run-tests.sh run while preparing the 0.1.32 release (#68). The bug is in the tests. The runtime behaves correctly.

1. They fail at random

tests/worker_reliability_test.rs:89: Activity should not have completed yet

What the test does:

1. Start the orchestration
2. Poll history every 5 ms until ActivityScheduled appears
3. Assert ActivityCompleted is not in history yet   ← nothing guarantees this
4. Shut down the runtime (the "crash"), start a new one, wait for completion

The activity returns at once. The runtime picks up the activity, and then its completion, on its next dispatcher polls (by default every 100 ms, with 2 orchestration slots and 2 worker slots). When both pickups happen right away, ActivityCompleted is saved before the test's first poll sees ActivityScheduled. In the failed run, the orchestration turn that scheduled the activity and the turn that received its result were 4.7 ms apart.

A scratch copy of the test's first phase shows the result depends only on how fast the runtime picks up work:

Dispatcher poll interval Test's first history poll Assertion fails
100 ms (default, as in the test) at once 1 of 30 runs
1 ms at once 25 of 30 runs
1 ms after 50 ms 30 of 30 runs

multiple_activities_reliability_after_crash makes the same kind of assertion ("No activities should have completed yet") and has the same race.

2. They never test a crash before the completion is saved

The "crash" is rt1.shutdown(None). That is a graceful shutdown: it sets a flag and waits 1 s. Worker slots stop fetching new work, but an activity that is already running finishes and saves its result. So at the "crash", the activity has either:

  • not been picked up yet, so phase 2 runs it for the first time, or
  • already finished and saved its result, so phase 2 only processes that result.

The case in the first test's name, "the activity ran but its completion was never saved", never happens.

3. They set an environment variable that nothing reads

Both tests set DUROXIDE_SQLITE_LOCK_TIMEOUT_MS=1000 so that redelivery after the crash is fast. Nothing in src/ has read that variable since a261621 (October 2025). The real worker lock timeout is the 30 s default (RuntimeOptions::worker_lock_timeout).

Proposed fix

Hold the activity in phase 1, so the crash really happens while it runs:

Phase 1 (first runtime, worker_lock_timeout = 2 s):
1. Activity: record "started", then never return
2. Test waits for "started" instead of polling history
3. Assert: ActivityScheduled is in history, ActivityCompleted is not
   → always true now: the activity is still running
4. Shut down the runtime (the "crash")
   → the lock-renewal task stops at shutdown; the activity never returns

Phase 2 (second runtime, same store):
5. Activity: record "started", return at once
6. The phase-1 lock expires within a few seconds → the work item is delivered again
7. Assert: the orchestration completes, and the activity started twice
  • Set the lock timeout through RuntimeOptions (worker_lock_timeout and worker_lock_renewal_buffer), and remove the environment variable.
  • multiple_activities_reliability_after_crash: same change, with worker_concurrency: 3 so all three activities start in phase 1. Each activity should start twice.
  • Don't use FailingProvider to fail every ack in phase 1. Each failed ack counts as an attempt, so with the default max_attempts of 10 the activity becomes a poison message within about a second.

Relevant code

  • tests/worker_reliability_test.rs: the two tests
  • tests/common/mod.rs: wait_for_history, which polls every 5 ms
  • src/runtime/dispatchers/worker.rs: the worker loop, run_activity_with_cancellation, and the lock-renewal task, which stops at shutdown
  • src/runtime/mod.rs: Runtime::shutdown and the RuntimeOptions defaults
主要言語
Rust
スター
226
フォーク
62
平均マージ
20時間 46分
マージ済み PR（30日）
6

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

microsoft/duroxide のほかの issue

microsoft/duroxide の issue をすべて見る

似ている issue

Rust の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。