`test_is_python_process_non_python_subprocess` always times out when `sleep` is a multicall binary (Alpine/BusyBox, RHEL UBI9)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- linux, python
- Domain
- operating-systems, testing
Research direction
Read Lib/test/test_profiling/test_sampling_profiler/test_children.py, especially _wait_for_process_exec() and test_is_python_process_non_python_subprocess. The issue includes a proposed check that detects when the child executable changes, regardless of its name. Update the helper and its call site, then run the named test on Linux; it should pass with multicall sleep binaries as well as regular binaries.
Written by the indexing model from the issue text.
Description
Bug report
Bug description:
The fix for gh-159534 (GH-160028, backported to 3.15 in GH-160080) makes test_is_python_process_non_python_subprocess fail every time on systems where sleep is provided by a multicall binary. That includes the Alpine buildbots, where the original gh-159534 failure was found:
FAIL: test_is_python_process_non_python_subprocess (test.test_profiling.test_sampling_profiler.test_children.TestIsPythonProcess.test_is_python_process_non_python_subprocess)
Test that a non-Python subprocess is not detected as Python.
----------------------------------------------------------------------
Traceback (most recent call last):
File ".../Lib/test/test_profiling/test_sampling_profiler/test_children.py", line 628, in test_is_python_process_non_python_subprocess
_wait_for_process_exec(proc, "sleep", SHORT_TIMEOUT)
File ".../Lib/test/test_profiling/test_sampling_profiler/test_children.py", line 91, in _wait_for_process_exec
for _ in sleeping_retry(timeout, max_delay=0.1):
File ".../Lib/test/support/__init__.py", line 2779, in sleeping_retry
for _ in busy_retry(timeout, err_msg, error=error):
File ".../Lib/test/support/__init__.py", line 2749, in busy_retry
raise AssertionError(msg)
AssertionError: timeout (45.1 seconds)
Buildbots
AMD64 Alpine Linux NoGIL 3.x:
- build 3444 (before GH-160028): the original intermittent failure,
True is not false. - build 3445 (first build with GH-160028):
timeout (45.1 seconds). Every build since fails the same way, e.g. build 3523.
AMD64 Alpine Linux 3.15:
- build 800 (before GH-160080): only
test_lazy_importfails. - build 815 (first build with GH-160080) and every build since, including build 851 just before v3.15.2 was tagged: this test fails too.
This failure is separate from the test_lazy_import crash on Alpine (gh-159535). The Alpine builders will stay red after that crash is fixed.
Cause
The new _wait_for_process_exec() helper waits until os.path.basename(os.readlink(f"/proc/{pid}/exe")) == "sleep". When sleep is a multicall binary, /proc/{pid}/exe points to the multicall binary instead:
| System | /usr/bin/sleep or /bin/sleep |
/proc/{pid}/exe of a running sleep |
|---|---|---|
| Alpine 3.24 (BusyBox) | symlink to /bin/busybox |
/bin/busybox |
RHEL UBI9 (coreutils-single, the image default) |
shebang script #!/usr/bin/coreutils --coreutils-prog-shebang=sleep |
/usr/bin/coreutils |
RHEL UBI9 after dnf swap coreutils-single coreutils |
regular binary | /usr/bin/sleep |
The condition never becomes true, so the test always times out.
Reproducer, tested in python:3-alpine and registry.access.redhat.com/ubi9/ubi:
import os, subprocess, time
proc = subprocess.Popen(["sleep", "10"])
time.sleep(0.5)
print(os.readlink(f"/proc/{proc.pid}/exe"))
proc.kill()
Suggested fix
The helper only needs to know that the child has called exec(). The program name doesn't matter. Before exec(), the forked child's /proc/{pid}/exe is the parent's Python executable, so wait until it differs from the parent's:
-def _wait_for_process_exec(proc, name, timeout):
+def _wait_for_process_exec(proc, timeout):
# Wait until a child process calls exec() to replace the forked process
- # executable. Raise AssertionError on timeout.
+ # executable. Don't check the executable name: it can be a multicall
+ # binary such as /bin/busybox. Raise AssertionError on timeout.
+ parent_exe = os.readlink("/proc/self/exe")
for _ in sleeping_retry(timeout, max_delay=0.1):
exe = os.readlink(f"/proc/{proc.pid}/exe")
- if os.path.basename(exe) == name:
+ if exe != parent_exe:
break
- _wait_for_process_exec(proc, "sleep", SHORT_TIMEOUT)
+ _wait_for_process_exec(proc, SHORT_TIMEOUT)
I tested this check with the reproducer on Alpine (BusyBox) and on UBI9 with both coreutils-single and coreutils. It detects the exec() in all three cases, while the current name check fails on Alpine and on UBI9 with coreutils-single.
CPython versions tested on:
3.15.2, CPython main branch
Operating systems tested on:
Linux (Alpine 3.24 buildbots, RHEL UBI9 container)
CPython versions tested on:
3.15
Operating systems tested on:
Linux
Linked PRs
- gh-160901
- Dominant language
- Python
- Stars
- 77.8k
- Forks
- 37.7k
- Avg merge
- 11h 25m
- Merged PRs (30d)
- 723
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from python/cpython
-
Null dereferencing of a pointer in initconfig.cPossibly taken @lazerg claimed this 1 day ago. Openinterpreter-core topic-C-API type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
python/cpython#160962 · 4 comments ·
Maintainers usually reply within 1 day
-
3.16 build pending type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
python/cpython#160829 · 5 comments ·
Maintainers usually reply within 1 day
-
dependencies infra topic-ensurepip
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
-
symtable: repr() of a Symbol raises TypeError under -X lazy_imports=allPossibly taken @anistark claimed this 2 days ago. Openstdlib topic-lazy-imports type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
python/cpython#160810 · 4 comments ·
Maintainers usually reply within 1 day
-
tarfile: extractfile(member).read() of a sparse member is slow due to repeated bytes concatenationPossibly taken @devdavidejesus claimed this 2 days ago. Openperformance stdlib triaged type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
python/cpython#160774 · 2 comments · 1 assignee ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
namespace operations
Difficulty 1/5 Under an hour Newbie friendliness 65/100
EclipseFdn/open-vsx.org#15057 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
FinanceFlash/unvibecode#349 ·
-
Mission statement: why this is free, in the README, Why This Exists and the wiki HomePossibly taken @ChiefGyk3D claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
ChiefGyk3D/Clouds-Against-Humanity#348 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
vllm-project/vime#508 ·
Maintainers usually reply within 4 days