add a hook that runs before ALL test projects / test modules and after all test projects / test modules

Aperta
  • Qualcosa non va?
#11,500 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
csharp
Ambito
devtools, testing-qa

Direzione di ricerca

Start by tracing how dotnet test, dotnet run --project, and server mode discover and execute test projects or modules. Compare where a session-wide lifecycle hook could be introduced across those entry points. Done means defining and validating a design that runs the before and after hooks exactly once for the whole test session, including parallel projects.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area/mtp needs/triage

Summary

currently at the moment some stuff is extremly hard with the M.T.P or even before. and that is running things before all tests projects did run and also after all test projects did run.

the problem is even amplified because its now possible to run tests in multiple ways, there is:

  • dotnet test
  • dotnet run --project TestProj.csproj
  • server mode

probably even more cases i do not now about.

now if you do not have a single csproj that will run as a cordinater you are basically in a game over statement if developers use either one and you want to create a test solution that basically warms stuff up or builds grouped reports at the end.
its also impossible to correctly build solutions to limit parallel execution of tests since, test projects run in parallel and tests inside the projects can also run in parallel. in a big monorepo that gets ugly insanly fast.

things that are also not possible because that feature is missing is test sharding, if a global hook would exists, it might even be possible to provide a way to introduce new args that would allow these features.

also running a single testcontainer, for example a database is basically impossible, running all migrations once before all tests, basically impossible, you basically have no concurrency control at all.

there can be a "hacky" solution for the "start", but not for the finish. the start can be done by creating a file level lock (urgh) the end can be done with a file level lock, if you only need to merge files, but that breaks for stopping containers.

Background and Motivation

we basically want more concurrency control and starting containers and migrations once for the whole test session no matter how many test projects we have.

Proposed Feature

add a way of providing a before all/after all hook, that does run exactly once, but not twice, if you have multiple projects that contain tests. maybe even provide a way to announce possible new arguments, however that would be hard if there are multiple extensions that provide the same arg, so that would need further design work.

the hook should run before anything, maybe for the server mode it can be done when the server mode is launched, its not like test containers would need to be reloaded all the time.

Alternative Designs

provide a solution to stop running test projects in parallel, however that would fix some issues it would mean that we need to pay the bootup multiple times.

Lingua principale
C#
Stelle
1.1k
Fork
312
Merge medio
11h 6m
PR unite (30g)
326

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/testfx

Tutte le issue di microsoft/testfx

Issue simili

Altre issue su C#

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.