Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[BUG] resolve failing and flaky tests in CI, and race conditions in tests using server

Abierto
#1,586 4 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Error
Claridad
Necesita aclaración
Estado de actividad
Tranquilo
Stack tecnológico
python
Área
backend, ci-cd, testing

Línea de trabajo

Start by reproducing the reported flaky failures in remote CI and inspect the server tests for shared-state or parallel-run races. Review the extension pattern from issue 1575 and compare the proposed mock, local-server, and decoupled server-test approaches. Done means the CI test matrix is reliable and server tests no longer fail sporadically when runs overlap.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug testing

Currently, the remote CI is not reliable and will result in faliures and sporadic failures that are unrelated for the PR in question - that means the remote CI is not usable for practical purposes. Problems:

  • it may still be that not all failures are temporarily decorated with xfail and skip properly
  • I think there is a even race condition in tests for the server, which results in sporadic failures, especially if multiple CI runs happen in parallel (which happens in every PR since there is a test matrix)

I am not sure how to address this best long-term; short-term I would suggest to simply skip all the flaky tests.

Mid-term, I think the server tests need to be decoupled from openml-python SDK tests, i.e., replaced by three things:

  • mock backends for simple API tests
  • a local server option that runs on the VM. This can follow the extension pattern introduced in the v1-v2 migration, see https://github.com/openml/openml-python/issues/1575
  • tests for the server API decoupled from tests from openml-python, possibly on a CRON and controlling for race conditions
Lenguaje dominante
Python
Estrellas
361
Forks
296
Sin PR fusionados en 30 d

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de openml/openml-python

Todos los issues de openml/openml-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.