Métricas do repositório

Stars: (5.625 stars)
Métricas de merge de PR: (Mesclagem média 13d 10h) (8 fundiu PRs em 30d)

Description

This is an umbrella issue to track progress making the Sphinx test suite reliable. That means:

No flaky tests.
Tests can run in parallel.
Tests are independent of each other.

Regarding the last item: test independence is important to ensure that each test is checking the behaviours it claims to, and does not inadvertently rely on the side-effects of some other test. Similarly, a test must not begin failing if any other test is run before it -- if it does, that could mean (but does not definitely mean) that the application's behaviour itself may vary based on the order in which code is evaluated.

It might seem like there are a large number of items here! However, please bear in mind that we have a total of more than 2000 individual pytest test cases that run.

Test flakiness

test_build_linkcheck.test_connect_to_selfsigned_fails (pr: #12166)
test_build_linkcheck.test_defaults (pr: #12166)
test_build_linkcheck.test_linkcheck_exclude_documents (pr: #12189)
test_numfig_disabled_warn (pr: #14107)
test_n_slowest_value (pr: #14110)
... report another?

Test parallelization

tests.test_builders.test_build_linkcheck (pr: #12126)
... report another?

Test independence

tests.test_builders.test_build_text.py (pr: #12868)
tests.test_domains.test_domain_cpp (pr: #12187)
tests.test_builders.test_build_html (pr: #12118)
tests.test_builders.test_build_html_download (pr: #12119)
... report another?

Guia do colaborador

Direção de pesquisa: Escolha um dos testes instáveis abertos (por exemplo, test numfig disabled warn), reproduza a instabilidade localmente executando o teste várias vezes e identifique a causa raiz. Em seguida, proponha uma correção que torne o teste determinístico e independente.
Pilha de tecnologia: python
Domain: testing
Tipo Issue: Teste
Difficulty: 3
Tempo estimado: 1-2 dias
Status da atividade: Ativo
Clarity: Claro
Prerequisites: Pythonpytest
Simpatia para novatos: 40