test_forkserver_sigkill: join() can return before the child sets the event
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 71/100
Research direction
Read Lib/test/_test_multiprocessing.py, especially check_forkserver_death() and test_forkserver_sigkill; the failure is that join() can return before the child sets the event after the forkserver is killed. Update the event check to wait with a timeout, then run the focused forkserver test. Done means the test allows the child time to set the event and still checks the expected exit code.
Written by the indexing model from the issue text.
Description
Bug report
Bug description:
test_forkserver_sigkill failed once during a full ./python.exe -m test -j0 run on my machine (macOS 27.0.1, Apple M3, pydebug build of main at 1705e512b30). The load average was around 65 at the time. It passes when run on its own.
test test.test_multiprocessing_forkserver.test_processes failed -- Traceback (most recent call last):
File "Lib/test/_test_multiprocessing.py", line 1007, in test_forkserver_sigkill
self.check_forkserver_death(signal.SIGKILL)
File "Lib/test/_test_multiprocessing.py", line 997, in check_forkserver_death
self.assertTrue(evt.is_set())
AssertionError: False is not true
(paths shortened)
This is the same failure as gh-78221 (bpo-34040). It was worked around in 2018 by shortening the child's delay relative to the parent's sleep (07888e1cce), but the race is still there.
check_forkserver_death() kills the forkserver right after starting the first process. After a SIGKILL, the forkserver can no longer report that child's exit status: proc.join() gets EOF on the status pipe, and Popen.poll() in Lib/multiprocessing/popen_forkserver.py sets returncode to 255 immediately. The child may still be running at that point. The test then checks evt.is_set(), which only passes if the child has set the event by then (within the 1 second sleep plus the time proc2 takes). The next line already accepts exit code 255 for the case where the forkserver was killed.
SIGINT is not affected: the forkserver ignores it, stays alive and reports the real exit code.
To reproduce it reliably, stop the first child with SIGSTOP for a few seconds after killing the forkserver:
Reproducer
import multiprocessing as mp, os, signal, threading, time
from multiprocessing.forkserver import _forkserver
def sleep_and_set(evt, delay=0.0):
time.sleep(delay)
evt.set()
if __name__ == '__main__':
mp.set_start_method('forkserver')
_forkserver.ensure_running()
evt = mp.Event()
proc = mp.Process(target=sleep_and_set, args=(evt, 0.5))
proc.start()
os.kill(_forkserver._forkserver_pid, signal.SIGKILL)
# Simulate the child not getting CPU time for 3 seconds.
os.kill(proc.pid, signal.SIGSTOP)
threading.Timer(3.0, os.kill, (proc.pid, signal.SIGCONT)).start()
time.sleep(1.0)
proc.join()
print(proc.exitcode, evt.is_set()) # 255 False
print(evt.wait(60)) # True: the child did run, later
Waiting for the event with a timeout, instead of checking it right after join(), fixes it.
The same code is in 3.13, 3.14 and 3.15.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS, Linux
Linked PRs
- gh-160296
- Dominant language
- Python
- Stars
- 77.8k
- Forks
- 37.7k
- Avg merge
- 12h 14m
- Merged PRs (30d)
- 729
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from python/cpython
-
tests topic-profiling type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
python/cpython#160881 · 1 comment ·
Maintainers usually reply within 1 day
-
3.16 build pending type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
python/cpython#160829 · 5 comments ·
Maintainers usually reply within 1 day
-
dependencies infra topic-ensurepip
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
-
symtable: repr() of a Symbol raises TypeError under -X lazy_imports=allPossibly taken @anistark claimed this 1 day ago. Openstdlib topic-lazy-imports type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
python/cpython#160810 · 4 comments ·
Maintainers usually reply within 1 day
-
tarfile: extractfile(member).read() of a sparse member is slow due to repeated bytes concatenationPossibly taken @devdavidejesus claimed this 1 day ago. Openperformance stdlib triaged type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
python/cpython#160774 · 2 comments · 1 assignee ·
Maintainers usually reply within 1 day
Similar issues
-
ai_p2 bug comp-datalake
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
ClickHouse/ClickHouse#128306 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
[bug] openlit: LLM spans from openlit 1.37+ get no OpenInference attributes, and cache and reasoning token counts are droppedPossibly taken @sohom-cs claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Arize-ai/openinference#4199 · 1 reaction ·
Maintainers usually reply within 2 days
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
MichaelYcJo/SpecSeal#1192 ·