debugpy listen silently crashing
@debonte 已经在做这个了。
开始于 2024年11月28日。
评估
这个 Issue 还没有评估数据。
描述
Environment data
- debugpy version: 1.8.8
- OS and version: A k8s pod running an Ubuntu 20.04.6 based container
- Python version (& distribution if applicable, e.g. Anaconda): 3.9
- Using VS Code or Visual Studio: VS Code
Actual behavior
I'm using the Ray Distributed Debugger (their code here) with Ray on K8S. It runs debugpy.listen , but when I check the port on which it listens, nothing is bound to that port (sudo lsof -i :$LISTEN_PORT). I enabled DEBUGPY_LOG_DIR to get more detailed logs, and I noticed that debugpy.pydevd.NNNN.log contains this near the end, indicating that it indeed crashed:
Traceback (most recent call last):
File "/my_app/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 422, in _on_run
cmd.send(self.sock)
File "/my_app/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_net_command.py", line 109, in send
sock.sendall(as_bytes)
BrokenPipeError: [Errno 32] Broken pipe
I looked in the issue trackers of debugpy, pydevd, and ray, and did some googling, and couldn't find much unfortunately. The only thing I found is that this may point to the connection between the local services (there is a client, server, and "debug server" and some incoming client (?) involved in running debugpy on the application side, it seems) breaking. I found this snippet in debugpy.adapter.NNNN.log:
I+00000.071: Listening for incoming Client connections on 10.40.0.130:51507...
I+00000.071: Listening for incoming Server connections on 127.0.0.1:39415...
I+00000.071: Sending endpoints info to debug server at localhost:60997:
{
"client": {
"host": "10.40.0.130",
"port": 51507
},
"server": {
"host": "127.0.0.1",
"port": 39415
}
}
I+00000.076: Accepted incoming Server connection from 127.0.0.1:43864.
Lastly, I noticed this in debugpy.{adapter,server}.NNNN.log but that seems to be ok, as I also saw this in healthy local runs:
I+00000.049: Error while enumerating installed packages.
Traceback (most recent call last):
File "/my_app/debugpy/adapter/../../debugpy/common/log.py", line 362, in get_environment_description
report(" {0}=={1}\n", pkg.name, pkg.version)
AttributeError: 'PathDistribution' object has no attribute 'name'
Stack where logged:
File "/my_app/python3_x86_64/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/my_app/python3_x86_64/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/my_app/debugpy/adapter/__main__.py", line 227, in <module>
main(_parse_argv(sys.argv))
File "/my_app/debugpy/adapter/__main__.py", line 50, in main
log.describe_environment("debugpy.adapter startup environment:")
File "/my_app/debugpy/adapter/../../debugpy/common/log.py", line 372, in describe_environment
info("{0}", get_environment_description(header))
File "/my_app/debugpy/adapter/../../debugpy/common/log.py", line 364, in get_environment_description
swallow_exception(
File "/my_app/debugpy/adapter/../../debugpy/common/log.py", line 215, in swallow_exception
_exception(format_string, *args, **kwargs)
All of this crashes already before I try connecting to the debugger.
I was also able to reproduce this without using Ray Distributed Debugger. I just connect to the k8s pod, create a small python script:
import debugpy
debugpy.listen(5678)
print("before wait_for_client")
debugpy.wait_for_client()
print("after wait_for_client")
print("before breakpoint")
debugpy.breakpoint()
print("after breakpoint")
Run it and check the log files and see the same crash happening (BrokenPipeError: [Errno 32] Broken pipe) in the pydevd logs.
When I run all of this locally, everything works fine. When running on ray on k8s, I run into this issue...
These are the full, lightly redacted, logs:
Questions:
- Is there a way to detect a crashed listen from code? If so, how?
- Any ideas on what makes this crash?
Expected behavior
Accepting oncoming connections on the debugpy.listen endpoint.
Steps to reproduce:
I'm afraid it will be hard to reproduce this in an environment other than our "ray on k8s" setup. But details are in the "Actual behavior" section.
- 主要语言
- Python
- 星标
- 2.5k
- 派生
- 208
- 平均合并
- 1 天 43 分钟
- 30 天内合并 PR
- 6
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
microsoft/debugpy 的其他 Issue
-
pydevd_comm.py start_client() does not set TCP_NODELAY on the connections it opens可能已有人在做 @nightcityblade 于 56 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
-
Document "rules"-option可能已有人在做 @nightcityblade 于 44 天前认领。 未关闭documentation
难度 2/5 1-3 小时 新手友好度 68/100
-
wx.EventLoop() deprecation warning可能已有人在做 @nightcityblade 于 43 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
-
debugpy.configure(python=) is not properly documented可能已有人在做 @nightcityblade 于 54 天前认领。 未关闭documentation
难度 2/5 1-3 小时 新手友好度 62/100
-
难度 5/5 一周以上 新手友好度 15/100
查看 microsoft/debugpy 的全部 Issue
相似的 Issue
-
enhancement good first issue
难度 2/5 1-3 小时 新手友好度 78/100
-
难度 2/5 1-3 小时 新手友好度 78/100
hatchet-dev/hatchet#5179 ·
维护者通常 1 天内回复
-
python-version
难度 1/5 1 小时以内 新手友好度 88/100
-
bug
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 1 天内回复
-
bug javascript P2-medium python release:v3.1
难度 2/5 1-3 小时 新手友好度 68/100
adrirubio/claude-deck#546 ·
维护者通常 1 天内回复