Tests flaking on CI since mid-August, on main and on PRs
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 35/100
Línea de trabajo
Start with the named tests in test/unit/worker_test.rb, test/integration/fiber_processes_lifecycle_test.rb, and the concurrency and lifecycle test files. Reproduce failures in the affected Rails and database CI combinations, then address the remaining failures one by one. Done means the listed tests pass reliably across the relevant CI matrix without unrelated regressions.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Since the 1.7.0 work landed, CI on main has been red on most pushes, and green PRs get failures unrelated to their changes. This is the catalogue of what failed between 2026-08-18 and 2026-08-29, grouped by test. Docs-only PRs (#789, #795) are included, so none of these come from the changes under test. Jobs that failed at "Initialize containers" or "Setup Ruby" (three runs) are infra and not counted.
1. WorkerTest#test_worker_keeps_running_after_a_regular_job_failure (test/unit/worker_test.rb:169) — the most frequent one, and not actually a flake: the test asserts JobBuffer.last_value == "ok", but RaisingJob also writes to JobBuffer when it raises, and both jobs run concurrently in the pool, so the last value depends on which finishes last. Failed on 4 jobs in the #795 merge run and 3 in the #768 merge run. Fix in #796.
2. WorkerTest#test_claim_and_process_more_enqueued_jobs_than_the_pool_size_allows_to_process_at_once (worker_test.rb:45), thread and fiber variants — fewer completed jobs than expected after wait_for_jobs_to_finish_for(2.seconds): Expected: 3 / Actual: 1 in thread mode (3.2 / mysql / rails_7_2, #789 run); Expected: 5 / Actual: 3 in fiber mode (3.4 / mysql / rails_8_0, #790 run).
3. FiberProcessesLifecycleTest#test_quit_supervisor_while_there_are_jobs_in-flight (test/integration/fiber_processes_lifecycle_test.rb:54) — three different assertions failing: the supervisor still alive after QUIT (line 74, 3.2 / postgres / rails_7_1, #790 run); the "no pause" job not completed (line 70, 3.4 / postgres / rails_7_1, same run); line 68 on 3.3 / mysql / rails_main (#768 merge run).
4. ConcurrencyControlsTest — #test_run_several_jobs_over_the_same_record_limiting_concurrency (line 98, Expected: "sCcC" / Actual: "sAcAsEcEsFcFsGcGsHcH", 3.4 / mysql / rails_8_0, #789 run) and #test_run_several_jobs_over_the_same_record_sequentially,_with_some_of_them_failing (line 112, 3.3 / postgres / rails_7_1, #790 run; line 114, 4 / postgres / rails_7_1, 1.7.0 bump run).
5. ForkedProcessesLifecycleTest — #test_kill_worker_individually (Timeout::Error: execution expired, 3.3 / postgres / rails_7_1, #791 merge run; line 265, 3.2 / postgres / rails_7_1, 1.7.0 bump run) and #test_term_supervisor_exceeding_timeout_while_there_are_jobs_in-flight (line 145, 3.4 / postgres / rails_7_1, #768 merge run).
6. LifecycleHooksTest — #test_run_lifecycle_hooks (line 89, 3.2 / postgres / rails_7_1, 1.7.0 bump run) and #test_handle_errors_on_lifecycle_hooks (line 114, 4 / postgres / rails_7_1, #778 merge run).
7. One-offs — ReadyExecutionTest#test_discard_all_by_queue (line 216, Expected: -1 / Actual: 0, 3.3 / mysql / rails_main, #789 run); JobsLifecycleTest#test_enqueue_and_run_jobs_that_fail_and_it's_discarded (line 100, 4 / mysql / rails_7_1, #790 run); SolidQueue::ClaimedExecutionTest#test_fail_with_error erroring with NoMethodError: undefined method 'prepare_for_execution' for nil from prepare_and_claim_job (3.4 / sqlite / rails_main, #789 run).
Postgres on Rails 7.1 shows up disproportionately in the lifecycle and concurrency ones, most of the rest are on MySQL, and SQLite is mostly fine apart from the ClaimedExecutionTest one. Rather than keep re-running jobs, let's fix these one by one. #602 was the previous catch-all for flaky tests and is closed in favour of this one.
Update 2026-08-29, later — two more on the #796 run, a one-line test change: ConcurrencyControlsTest#test_discard_on_conflict_and_release_semaphore (line 266, Expected: "s1c1s3c3" / Actual: "s1c1", 3.2 / postgres / rails_7_1, so group 4 and the same combo again) and FiberProcessesLifecycleTest#test_term_supervisor_while_there_are_jobs_in-flight (Expected: ["background"] / Actual: [], then Timeout::Error in teardown waiting for the supervisor to terminate, 4 / sqlite / rails_8_1, group 3's sibling). Note for anyone reading those logs: the ActiveRecord::StatementInvalid: connection lost / ConnectionNotEstablished: connection is closed lines printed mid-run come from mocha, i.e. from the tests that stub database failures on purpose to exercise the supervisor's resilience, not from the runner's database.
Update 2026-08-29, evening — from the first run of #798 (run), which fixes groups 2 and 3; none of these are in the tests it touches: AsyncProcessesLifecycleTest#test_term_supervisor_exceeding_timeout_while_there_are_jobs_in-flight (line 170, 3.4 / postgres / rails_7_1 — group 5's async sibling, and the interesting one: its own diagnostic assertion fired, showing the 32-second "pause" job with status: "completed" 1 ms after "no pause", which the test's comment already flags as impossible from within the test's own flow); SolidQueue::ProcessTest#test_prune_processes_with_expired_heartbeats_and_fail_claimed_executions (3.3 / mysql / rails_7_1, new to the list); PluginAsyncTest#test_stop_the_queue_on_puma's_restart (test/integration/puma/plugin_testing.rb:56, Expected: 1 / Actual: 0, 3.2 / sqlite / rails_8_0, new to the list). Plus group 1 once more, as that branch predates #796.
Update 2026-08-31 — one more for group 4, new test name: ConcurrencyControlsTest#test_rely_on_dispatcher_to_unblock_blocked_executions_with_an_expired_semaphore (line 174, Expected false to be truthy, 3.4 / mysql / rails_7_1, #801 run; the run's other 57 jobs passed). Separately, #801 explains and fixes the hangs in this catalogue (zombie processes on sqlite): killing a thread waiting in SQLite's busy handler leaks the connection mutex and deadlocks the process's at-exit finalization — forked processes now exit with exit! once their block completes.
- Lenguaje dominante
- Ruby
- Estrellas
- 2.5k
- Forks
- 252
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Preparar el entorno
Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.
- Incluye un Dockerfile o un archivo de Docker Compose
- Sin plantilla de pull request
- Sin guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de rails/solid_queue
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 42/100
rails/solid_queue#808 ·
-
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
rails/solid_queue#806 ·
-
Dificultad 3/5 1-2 días Aptitud para principiantes 65/100
rails/solid_queue#805 ·
-
limits_concurrency on_conflict: :discard looking only into running jobs and not blocked jobsAbierto
Dificultad 3/5 1-2 días Aptitud para principiantes 52/100
rails/solid_queue#804 · 1 comentario ·
-
Dificultad 3/5 1-2 días Aptitud para principiantes 35/100
rails/solid_queue#802 ·
Todos los issues de rails/solid_queue
Issues similares
-
Supported format or minItems skips transformation of sibling constraintsPosiblemente ocupada @sylvesterkaczmarek la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
anthropics/anthropic-sdk-ruby#253 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 90/100
forem/forem#23933 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 66/100
rubyevents/rubyevents#2170 ·
Los mantenedores suelen responder en 1 día
-
Allow customizing the page titleAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
rails/mission_control-jobs#347 ·
-
PROJECT: ILR/DRS
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
thewca/worldcubeassociation.org#15914 ·
Los mantenedores suelen responder en 1 día