test: flaky integration tests when GetRandomMovie() returns a soft-deleted movie

オープン 初心者向け
  • 何か問題がありますか？
#600 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

評価

難易度
2/5
見積もり時間
1〜3時間
初心者へのやさしさ
78/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
csharp
領域
testing

調査の方向性

2つの ServiceApplicationFactory.cs ファイルで指定されている GetRandomMovie() から始め、それぞれが有効な映画をどのように選択するかを比較してください。両方の実装で論理削除されたエントリを除外し、その後、該当するクライアントとサーバーの統合テストを実行してください。ランダム選択で論理削除された映画が返されず、それらのテストが成功すれば完了です。

索引モデルが issue の本文から書いたものです。

説明

Describe the bug

Several integration tests are intermittently flaky because GetRandomMovie() can return a movie that an earlier test in the same class has already soft-deleted and never restored.

Observed failure, from PR #599 (Build and Test Library, run 42093778343). The change in that PR is unrelated, and the identical commit passed on a fork run:

CommunityToolkit.Datasync.Client.Test.Service.Integration_Tests.Read_SoftDeleted_NotDeleted_Returns200 [FAIL]
  CommunityToolkit.Datasync.Client.DatasyncHttpException : Gone
    at ServiceResponse.ThrowIfNotSuccessful(...)
    at DatasyncServiceClient`1.GetAsync(String id, ...)
    at Integration_Tests.Read_SoftDeleted_NotDeleted_Returns200() line 152

Root cause

  1. Each test class that uses the in-memory test server creates one ServiceApplicationFactory via IClassFixture<ServiceApplicationFactory>. All tests in the class share one in-memory movie repository.
  2. GetRandomMovie() picks a random movie that satisfies IsValid(...) (title, year and duration). It does not exclude movies where Deleted == true.
  3. Several tests soft-delete a random movie, either with SoftDelete(existingMovie) or through DELETE api/in-memory/softmovies/{id}. They never restore it.
  4. If a later test in the same class randomly picks one of those movies and calls the softmovies endpoint, the server correctly returns 410 Gone (or the test's other expectations break), and the test fails.

The failure rate is low and depends on test ordering. A few leaked soft-deleted rows sit in the set of seeded valid movies, so it is probably in the low single-digit percent range per run. That is why it only appears occasionally in CI.

I could not reproduce it locally. 15 consecutive runs of Service.Integration_Tests all passed.

Affected tests

Any test that calls GetRandomMovie() and expects a non-deleted movie, in a class that also soft-deletes movies. The soft-deleting tests also mutate Deleted state themselves.

tests/CommunityToolkit.Datasync.Client.Test

  • Service/Integration_Tests.cs soft-deletes in Create_SoftDeleted_Returns409, Read_SoftDeleted_Deleted_Returns404, Read_SoftDeleted_Deleted_WithIncludeDeleted_Returns200, Delete_NotSoftDeleted_Returns204 and Delete_SoftDeletedId_Returns410. All 16 GetRandomMovie() call sites in that file are exposed.

tests/CommunityToolkit.Datasync.Server.Test has the same GetRandomMovie() implementation in Helpers/ServiceApplicationFactory.cs, and the same soft-delete pattern:

  • Service/Read_Tests.cs: Read_SoftDeleted_Deleted_Returns410 and Read_SoftDeleted_Deleted_WithIncludeDeleted_Returns200. Read_SoftDeleted_NotDeleted_Returns200 has the same signature as the observed failure.
  • Service/Delete_Tests.cs: Delete_NotSoftDeleted_Returns204 and Delete_SoftDeletedId_Returns410.
  • Service/Replace_Tests.cs: three soft-delete tests.
  • Service/Create_Tests.cs: one soft-delete test.

Suggested fix

Smallest fix: exclude soft-deleted movies in both GetRandomMovie() implementations:

repository.GetEntities().Where(x => IsValid(x) && !x.Deleted).ToList();
  • tests/CommunityToolkit.Datasync.Client.Test/Helpers/ServiceApplicationFactory.cs
  • tests/CommunityToolkit.Datasync.Server.Test/Helpers/ServiceApplicationFactory.cs

Tests that need a deleted movie already call SoftDelete(...) explicitly, so they are unaffected.

A more robust option is to also restore or reset the repository per test, as ResetInMemoryMovies() already does for the offline push tests. That removes the cross-test state sharing entirely, at some cost in test runtime.

Additional context

Delete_* tests that hard-delete a movie remove it from the repository, so they don't cause this problem. Only soft-deleted rows that stay in the repository do.

主要言語
C#
スター
184
フォーク
48
平均マージ
21分
マージ済み PR（30日）
7

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

CommunityToolkit/Datasync のほかの issue

CommunityToolkit/Datasync の issue をすべて見る

似ている issue

C# の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。