Support debugging of indirect child processes.
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start by reviewing the mentioned debugpy.server.cli.options, debugpy.server.api, and debugpy.connect entry points, then compare them with the existing direct-child multiprocess support. Define the official API boundary for carrying session context through indirect non-Python processes; done means the shown multi-level worker tree can connect without relying on implementation details.
Written by the indexing model from the issue text.
Description
While debugpy natively supports multiprocess debugging, this works only for directly launched child processes.
Sometimes, though, there may be several levels on non-Python processes between the main process and the child one needs to debug. I am hitting this limitation while trying to debug a user-defined function, that runs in the context of a PySpark worker process. The process tree looks like this:
<python process hosting my main program>
└ <pyspark process>
└ <yet another pyspark process>
├ <python worker process hosting the UDFs>
├ <python worker process hosting the UDFs>
├ ...
└ <python worker process hosting the UDFs>
After spelunking the innards of debugpy for a few hours, I've managed to come up with this code:
import pickle
import os
import debugpy
def capture_session_context():
"""To be called in the main process to capture context of the current debugpy session."""
options = debugpy.server.cli.options
context = pickle.dumps((options.address, options.adapter_access_token, os.getpid()), 0)
os.environ["DEBUGPY_CONTEXT"] = context.decode('ascii')
def connect_to_session():
"""To be called in a worker process to connect to the parent session."""
context = os.environ['DEBUGPY_CONTEXT'].encode('ascii')
address, token, ppid = pickle.loads(context)
from debugpy.server import api
api.pydevd.SetupHolder.setup = {"ppid": ppid}
debugpy.connect(address, access_token=token)
Obviously, this hack uses a bunch of implementation details. It would be nice if this scenario was officially supported.
(I mean exposing official APIs similar to the above, not that debugpy should support PySperk specifically.
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 202
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/debugpy
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
All issues in microsoft/debugpy
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
xinnan-tech/xiaozhi-fde-talk#263 ·
-
rules
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
huggingface/Repo2RLEnv#163 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 95/100
huggingface/sentence-transformers#4074 ·
-
comp/dashboard invalid P3
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
NousResearch/hermes-agent#121143 ·