test(e2e): align markers and execution entry points
I maintainer di solito rispondono entro 1 giorno
Una pull request collegata è già stata integrata.
- #559 di @yeongseon — integrata
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 58/100
Direzione di ricerca
Start by reading the Hatch task in pyproject.toml, the two modules under tests/e2e/, and the test command in .github/workflows/e2e-azure.yml; reproduce the reported collection failure with python -m pytest --collect-only -m e2e tests/e2e/ --no-cov. Update the marker and entry-point contract together, then run the focused regression check and confirm the selector collects 21 tests without changing default unit coverage. Check that missing E2E_BASE_URL gives a clear error for dedicated certification runs.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Context
Review of azure-functions-validation-python at commit b38d702adb1039ed1eda2ef3799d873af2506f41.
The supported make e2e-local and make e2e-azure entry points invoke this Hatch script:
e2e-azure = "pytest -v -m e2e tests/e2e/"
However, neither tests/e2e/test_validation_e2e.py nor tests/e2e/test_pipeline_branches_e2e.py applies the e2e marker. The latter's module-level pytestmark is only a skipif for a missing base URL. The test package and shared conftest do not add an E2E collection marker.
A selector-level reproduction from the repository root, using the same selector as the Hatch task and no Azure connection, is:
python -m pytest --collect-only -m e2e tests/e2e/ --no-cov
Observed result at the reviewed commit:
no tests collected (21 deselected) in 0.07s
exit code: 5
Expected behavior: the dedicated E2E entry points select the 21 intended real-host tests.
This does not mean the release workflow skipped those tests. Its current command is pytest tests/e2e -v --no-cov ..., without -m e2e. Since the tests are unmarked, that command currently runs them despite the default marker exclusions. The latest successful v0.16.0 release, run 38485749334 / Azure job 115599053509, recorded 21 passed in 12.65s on Python 3.12.15.
A marker-only fix would break the release command: default pytest options exclude e2e, and they also enforce 95% coverage of local package code. The Hatch and workflow entry points must be updated together, selecting E2E explicitly and disabling local source coverage for HTTP-only remote tests.
Acceptance Checklist
- Apply the
e2emarker consistently to both real-host modules while preserving their intended configuration checks. - Update the Hatch E2E task and Azure workflow command together to use
-m e2e --no-cov, or an equivalent explicit selector and coverage policy. - Verify that the dedicated selector collects the intended 21 tests rather than exiting with code 5.
- Keep real-host E2E tests excluded from the default unit suite and leave the unit-test 95% coverage policy unchanged.
- Ensure both Makefile entry points and the release workflow use the same E2E selection contract.
- Provide a clear error for a dedicated certification invocation with missing
E2E_BASE_URL, so an entirely skipped suite cannot be mistaken for a successful live verification. - Add a focused regression check for the marker/entry-point contract; it should not require provisioning Azure.
- Preserve the existing body/query/path/header, async, response serialization, sanitized error, and worker context coverage in the release E2E suite.
Out of scope
- Replacing the real Azure gate with import-only or mock-only tests.
- Claiming that the current successful release ran zero E2E tests.
- Altering runtime validation behavior or reducing the unit-test coverage threshold.
- Deploying additional Azure resources merely to reproduce the collection failure.
- Reviewing or changing E2E configuration in sibling repositories.
References
- Lingua principale
- Python
- Stelle
- 13
- Fork
- 4
- Merge medio
- 14h 46m
- PR unite (30g)
- 89
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
FinanceFlash/unvibecode#509 ·
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 82/100
FAIRmat-NFDI/nomad-distro-dev#161 ·
-
bug v2
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
modelcontextprotocol/python-sdk#3821 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
bug severity:low
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
I maintainer di solito rispondono entro 2 giorni
-
Log-odds calibration uses upper middle score instead of median for even candidate setsForse già presa @flysheep-ai l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 1 giorno