session: a job's stop waits for the log-retention sweep, and publishing its end first races folder teardown
Maintainer antworten meist innerhalb von 1 Tag
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 68/100
- Issue-Typ
- Bug
- Klarheit
- Klar beschrieben
- Aktivitätsstatus
- Aktiv
- Tech-Stack
- go
- Bereich
- backend, distributed-systems, testing-qa
Rechercherichtung
Start with job.settle and jobSink.close in internal/session/jobs.go, searching for retention, then read the sweep in internal/session/jobretention.go and the wait logic in internal/session/task_job_park_test.go. Run the two named tests with GOMAXPROCS=2 under load. Done means job stopping no longer waits on retention, observers cannot race folder teardown, retention and shutdown behavior remain tested, and the new folder-removal test passes.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
What happened
Since #1603 (log retention, in #1627), a job's ending runs the whole log-folder retention sweep before the job's status leaves running. Under load, a stopped job reads as running for 47–130 ms, against about 2 ms before. TestStopWorkDoesNotWakeAndOnlyFreshSubmissionRestarts fails 5 of 20 runs under load on #1627's head 6cc872c8e, and 0 of 20 on dev 837b2b06a.
The obvious fix, publishing the final status first and then sweeping, was tried on #1627 (8b0edfd6d) and reverted (2e007e33b). Anything that watches running() then removes the job's folder while the sweep is still writing in it. CI:
--- FAIL: TestAParkedWorkerIsHandedBackWithARecordBeforeItsWholeAllowance (1.05s)
testing.go:1464: TempDir RemoveAll cleanup: unlinkat …/003/.codeaf/jobs: directory not empty
waitSettled in internal/session/task_job_park_test.go polls !one.running(). Giving a task's copy back after its jobs end has the same shape in the product.
Replication
Deterministic (no model). On #1627's head, loaded (run beside another package's tests, or with GOMAXPROCS=2):
go test -run '^TestStopWorkDoesNotWakeAndOnlyFreshSubmissionRestarts$' -count=20 ./internal/session
A few runs fail. With 8b0edfd6d applied instead, run
-run '^TestAParkedWorkerIsHandedBackWithARecordBeforeItsWholeAllowance$' -count=50 under the same load. It fails on the TempDir cleanup.
Where
job.settleandjobSink.closeininternal/session/jobs.go(searchretention)- the sweep in
internal/session/jobretention.go
The fix
A job's stop is not slowed by retention, and no observer of a job's end can race the sweep. One way to get both: the sweep stops being part of any single job's ending. It runs as one registry-owned pass, which shutdown and the registry's close join, and which tolerates its directory disappearing. Test helpers that wait for a job's end wait on done, not on running().
Acceptance
- Unit:
TestStopWorkDoesNotWakeAndOnlyFreshSubmissionRestartspasses 50/50 underGOMAXPROCS=2with another package's tests running beside it. - Unit:
TestAParkedWorkerIsHandedBackWithARecordBeforeItsWholeAllowancepasses 50/50 under the same load. - Unit: a new test ends a job and immediately removes its jobs folder, as a copy's give-back does, and nothing is written into it afterwards.
- Unit: a new test proves retention still bounds the folder (the #1599 law) and that shutdown still joins a sweep in progress.
- Vorherrschende Sprache
- Go
- Sterne
- 115
- Forks
- 14
- Ø Merge
- 9 Std. 15 Min.
- Gemergte PRs (30 T.)
- 467
Entwicklungsumgebung
Die Einrichtungsdateien dieses Projekts haben wir noch nicht geprüft. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus Agent-Field/CodeAF
-
area:session bug sev:papercut
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 82/100
Agent-Field/CodeAF#1779 ·
Maintainer antworten meist innerhalb von 1 Tag
-
area:session bug sev:papercut
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
Agent-Field/CodeAF#1768 ·
Maintainer antworten meist innerhalb von 1 Tag
-
area:chat bug sev:papercut
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
Agent-Field/CodeAF#1742 ·
Maintainer antworten meist innerhalb von 1 Tag
-
area:build bug sev:papercut
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 88/100
Agent-Field/CodeAF#1679 ·
Maintainer antworten meist innerhalb von 1 Tag
-
area:session bug sev:serious
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 85/100
Agent-Field/CodeAF#1678 ·
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in Agent-Field/CodeAF
Ähnliche Issues
-
duplication
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
openvibely/openvibely#1443 ·
Maintainer antworten meist innerhalb von 2 Tagen
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 80/100
keyxmakerx/Chronicle#1179 ·
Maintainer antworten meist innerhalb von 1 Tag
-
raised-by:worker
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 68/100
medici-finance/assay#2486 ·
Maintainer antworten meist innerhalb von 1 Tag
-
area/testing kind/bug triage/needs-triage
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 75/100
cozystack/cozystack#4841 · 1 Reaktion ·
Maintainer antworten meist innerhalb von 2 Tagen
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
openimsdk/openim-sdk-core#1127 ·