test: flaky integration tests when GetRandomMovie() returns a soft-deleted movie

Aperta Adatta ai principianti
  • Qualcosa non va?
#600 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
78/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
csharp
Ambito
testing

Direzione di ricerca

Inizia da GetRandomMovie() nei due file ServiceApplicationFactory.cs indicati e confronta come ciascuno seleziona i film validi. Escludi le voci eliminate logicamente in entrambe le implementazioni, poi esegui i test di integrazione interessati del client e del server; il lavoro è completato quando la selezione casuale non può restituire un film eliminato logicamente e quei test hanno esito positivo.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug

Several integration tests are intermittently flaky because GetRandomMovie() can return a movie that an earlier test in the same class has already soft-deleted and never restored.

Observed failure, from PR #599 (Build and Test Library, run 42093778343). The change in that PR is unrelated, and the identical commit passed on a fork run:

CommunityToolkit.Datasync.Client.Test.Service.Integration_Tests.Read_SoftDeleted_NotDeleted_Returns200 [FAIL]
  CommunityToolkit.Datasync.Client.DatasyncHttpException : Gone
    at ServiceResponse.ThrowIfNotSuccessful(...)
    at DatasyncServiceClient`1.GetAsync(String id, ...)
    at Integration_Tests.Read_SoftDeleted_NotDeleted_Returns200() line 152

Root cause

  1. Each test class that uses the in-memory test server creates one ServiceApplicationFactory via IClassFixture<ServiceApplicationFactory>. All tests in the class share one in-memory movie repository.
  2. GetRandomMovie() picks a random movie that satisfies IsValid(...) (title, year and duration). It does not exclude movies where Deleted == true.
  3. Several tests soft-delete a random movie, either with SoftDelete(existingMovie) or through DELETE api/in-memory/softmovies/{id}. They never restore it.
  4. If a later test in the same class randomly picks one of those movies and calls the softmovies endpoint, the server correctly returns 410 Gone (or the test's other expectations break), and the test fails.

The failure rate is low and depends on test ordering. A few leaked soft-deleted rows sit in the set of seeded valid movies, so it is probably in the low single-digit percent range per run. That is why it only appears occasionally in CI.

I could not reproduce it locally. 15 consecutive runs of Service.Integration_Tests all passed.

Affected tests

Any test that calls GetRandomMovie() and expects a non-deleted movie, in a class that also soft-deletes movies. The soft-deleting tests also mutate Deleted state themselves.

tests/CommunityToolkit.Datasync.Client.Test

  • Service/Integration_Tests.cs soft-deletes in Create_SoftDeleted_Returns409, Read_SoftDeleted_Deleted_Returns404, Read_SoftDeleted_Deleted_WithIncludeDeleted_Returns200, Delete_NotSoftDeleted_Returns204 and Delete_SoftDeletedId_Returns410. All 16 GetRandomMovie() call sites in that file are exposed.

tests/CommunityToolkit.Datasync.Server.Test has the same GetRandomMovie() implementation in Helpers/ServiceApplicationFactory.cs, and the same soft-delete pattern:

  • Service/Read_Tests.cs: Read_SoftDeleted_Deleted_Returns410 and Read_SoftDeleted_Deleted_WithIncludeDeleted_Returns200. Read_SoftDeleted_NotDeleted_Returns200 has the same signature as the observed failure.
  • Service/Delete_Tests.cs: Delete_NotSoftDeleted_Returns204 and Delete_SoftDeletedId_Returns410.
  • Service/Replace_Tests.cs: three soft-delete tests.
  • Service/Create_Tests.cs: one soft-delete test.

Suggested fix

Smallest fix: exclude soft-deleted movies in both GetRandomMovie() implementations:

repository.GetEntities().Where(x => IsValid(x) && !x.Deleted).ToList();
  • tests/CommunityToolkit.Datasync.Client.Test/Helpers/ServiceApplicationFactory.cs
  • tests/CommunityToolkit.Datasync.Server.Test/Helpers/ServiceApplicationFactory.cs

Tests that need a deleted movie already call SoftDelete(...) explicitly, so they are unaffected.

A more robust option is to also restore or reset the repository per test, as ResetInMemoryMovies() already does for the offline push tests. That removes the cross-test state sharing entirely, at some cost in test runtime.

Additional context

Delete_* tests that hard-delete a movie remove it from the repository, so they don't cause this problem. Only soft-deleted rows that stay in the repository do.

Lingua principale
C#
Stelle
184
Fork
48
Merge medio
21m
PR unite (30g)
7

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di CommunityToolkit/Datasync

Tutte le issue di CommunityToolkit/Datasync

Issue simili

Altre issue su C#

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.