session: a job's stop waits for the log-retention sweep, and publishing its end first races folder teardown
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 68/100
- Tipo de issue
- Error
- Claridad
- Bien especificado
- Estado de actividad
- Activo
- Stack tecnológico
- go
- Área
- backend, distributed-systems, testing-qa
Línea de trabajo
Start with job.settle and jobSink.close in internal/session/jobs.go, searching for retention, then read the sweep in internal/session/jobretention.go and the wait logic in internal/session/task_job_park_test.go. Run the two named tests with GOMAXPROCS=2 under load. Done means job stopping no longer waits on retention, observers cannot race folder teardown, retention and shutdown behavior remain tested, and the new folder-removal test passes.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
What happened
Since #1603 (log retention, in #1627), a job's ending runs the whole log-folder retention sweep before the job's status leaves running. Under load, a stopped job reads as running for 47–130 ms, against about 2 ms before. TestStopWorkDoesNotWakeAndOnlyFreshSubmissionRestarts fails 5 of 20 runs under load on #1627's head 6cc872c8e, and 0 of 20 on dev 837b2b06a.
The obvious fix, publishing the final status first and then sweeping, was tried on #1627 (8b0edfd6d) and reverted (2e007e33b). Anything that watches running() then removes the job's folder while the sweep is still writing in it. CI:
--- FAIL: TestAParkedWorkerIsHandedBackWithARecordBeforeItsWholeAllowance (1.05s)
testing.go:1464: TempDir RemoveAll cleanup: unlinkat …/003/.codeaf/jobs: directory not empty
waitSettled in internal/session/task_job_park_test.go polls !one.running(). Giving a task's copy back after its jobs end has the same shape in the product.
Replication
Deterministic (no model). On #1627's head, loaded (run beside another package's tests, or with GOMAXPROCS=2):
go test -run '^TestStopWorkDoesNotWakeAndOnlyFreshSubmissionRestarts$' -count=20 ./internal/session
A few runs fail. With 8b0edfd6d applied instead, run
-run '^TestAParkedWorkerIsHandedBackWithARecordBeforeItsWholeAllowance$' -count=50 under the same load. It fails on the TempDir cleanup.
Where
job.settleandjobSink.closeininternal/session/jobs.go(searchretention)- the sweep in
internal/session/jobretention.go
The fix
A job's stop is not slowed by retention, and no observer of a job's end can race the sweep. One way to get both: the sweep stops being part of any single job's ending. It runs as one registry-owned pass, which shutdown and the registry's close join, and which tolerates its directory disappearing. Test helpers that wait for a job's end wait on done, not on running().
Acceptance
- Unit:
TestStopWorkDoesNotWakeAndOnlyFreshSubmissionRestartspasses 50/50 underGOMAXPROCS=2with another package's tests running beside it. - Unit:
TestAParkedWorkerIsHandedBackWithARecordBeforeItsWholeAllowancepasses 50/50 under the same load. - Unit: a new test ends a job and immediately removes its jobs folder, as a copy's give-back does, and nothing is written into it afterwards.
- Unit: a new test proves retention still bounds the folder (the #1599 law) and that shutdown still joins a sweep in progress.
- Lenguaje dominante
- Go
- Estrellas
- 115
- Forks
- 14
- Merge medio
- 11 h 32 min
- PR fusionados (30 d)
- 747
Preparar el entorno
Aún no hemos revisado los archivos de configuración de este proyecto. Empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de Agent-Field/CodeAF
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
Agent-Field/CodeAF#1679 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
Agent-Field/CodeAF#1678 ·
Los mantenedores suelen responder en 1 día
-
area:chat bug sev:papercut
Dificultad 2/5 1-3 horas Aptitud para principiantes 86/100
Agent-Field/CodeAF#1592 ·
Los mantenedores suelen responder en 1 día
-
area:chat feature
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
Agent-Field/CodeAF#1510 ·
Los mantenedores suelen responder en 1 día
-
area:tests bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Agent-Field/CodeAF#1489 ·
Los mantenedores suelen responder en 1 día
Todos los issues de Agent-Field/CodeAF
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 1 día
-
status: ready for dev
Dificultad 1/5 1-3 horas Aptitud para principiantes 92/100
hyperledger-labs/fabric-smart-client#2004 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
Los mantenedores suelen responder en 1 día
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
apache/datasketches-go#189 ·
Los mantenedores suelen responder en 1 día