test(e2e): align markers and execution entry points

Cerrado
  • ¿Algo no está bien?
#545 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Ya se ha fusionado un pull request relacionado.

  • #559 de @yeongseon — fusionado

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
58/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
azure, github-actions, python
Área
ci-cd, testing

Línea de trabajo

Start by reading the Hatch task in pyproject.toml, the two modules under tests/e2e/, and the test command in .github/workflows/e2e-azure.yml; reproduce the reported collection failure with python -m pytest --collect-only -m e2e tests/e2e/ --no-cov. Update the marker and entry-point contract together, then run the focused regression check and confirm the selector collects 21 tests without changing default unit coverage. Check that missing E2E_BASE_URL gives a clear error for dedicated certification runs.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

ci needs-triage priority:medium test

Context

Review of azure-functions-validation-python at commit b38d702adb1039ed1eda2ef3799d873af2506f41.

The supported make e2e-local and make e2e-azure entry points invoke this Hatch script:

e2e-azure = "pytest -v -m e2e tests/e2e/"

However, neither tests/e2e/test_validation_e2e.py nor tests/e2e/test_pipeline_branches_e2e.py applies the e2e marker. The latter's module-level pytestmark is only a skipif for a missing base URL. The test package and shared conftest do not add an E2E collection marker.

A selector-level reproduction from the repository root, using the same selector as the Hatch task and no Azure connection, is:

python -m pytest --collect-only -m e2e tests/e2e/ --no-cov

Observed result at the reviewed commit:

no tests collected (21 deselected) in 0.07s
exit code: 5

Expected behavior: the dedicated E2E entry points select the 21 intended real-host tests.

This does not mean the release workflow skipped those tests. Its current command is pytest tests/e2e -v --no-cov ..., without -m e2e. Since the tests are unmarked, that command currently runs them despite the default marker exclusions. The latest successful v0.16.0 release, run 38485749334 / Azure job 115599053509, recorded 21 passed in 12.65s on Python 3.12.15.

A marker-only fix would break the release command: default pytest options exclude e2e, and they also enforce 95% coverage of local package code. The Hatch and workflow entry points must be updated together, selecting E2E explicitly and disabling local source coverage for HTTP-only remote tests.

Acceptance Checklist

  • Apply the e2e marker consistently to both real-host modules while preserving their intended configuration checks.
  • Update the Hatch E2E task and Azure workflow command together to use -m e2e --no-cov, or an equivalent explicit selector and coverage policy.
  • Verify that the dedicated selector collects the intended 21 tests rather than exiting with code 5.
  • Keep real-host E2E tests excluded from the default unit suite and leave the unit-test 95% coverage policy unchanged.
  • Ensure both Makefile entry points and the release workflow use the same E2E selection contract.
  • Provide a clear error for a dedicated certification invocation with missing E2E_BASE_URL, so an entirely skipped suite cannot be mistaken for a successful live verification.
  • Add a focused regression check for the marker/entry-point contract; it should not require provisioning Azure.
  • Preserve the existing body/query/path/header, async, response serialization, sanitized error, and worker context coverage in the release E2E suite.

Out of scope

  • Replacing the real Azure gate with import-only or mock-only tests.
  • Claiming that the current successful release ran zero E2E tests.
  • Altering runtime validation behavior or reducing the unit-test coverage threshold.
  • Deploying additional Azure resources merely to reproduce the collection failure.
  • Reviewing or changing E2E configuration in sibling repositories.

References

Lenguaje dominante
Python
Estrellas
13
Forks
4
Merge medio
15 h 56 min
PR fusionados (30 d)
72

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de yeongseon/azure-functions-validation-python

Todos los issues de yeongseon/azure-functions-validation-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.