debugpy listen silently crashing
@debonte ya está trabajando en esto.
Desde el 28/11/2024.
Evaluación
Este issue todavía no se ha evaluado.
Descripción
Environment data
- debugpy version: 1.8.8
- OS and version: A k8s pod running an Ubuntu 20.04.6 based container
- Python version (& distribution if applicable, e.g. Anaconda): 3.9
- Using VS Code or Visual Studio: VS Code
Actual behavior
I'm using the Ray Distributed Debugger (their code here) with Ray on K8S. It runs debugpy.listen , but when I check the port on which it listens, nothing is bound to that port (sudo lsof -i :$LISTEN_PORT). I enabled DEBUGPY_LOG_DIR to get more detailed logs, and I noticed that debugpy.pydevd.NNNN.log contains this near the end, indicating that it indeed crashed:
Traceback (most recent call last):
File "/my_app/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 422, in _on_run
cmd.send(self.sock)
File "/my_app/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_net_command.py", line 109, in send
sock.sendall(as_bytes)
BrokenPipeError: [Errno 32] Broken pipe
I looked in the issue trackers of debugpy, pydevd, and ray, and did some googling, and couldn't find much unfortunately. The only thing I found is that this may point to the connection between the local services (there is a client, server, and "debug server" and some incoming client (?) involved in running debugpy on the application side, it seems) breaking. I found this snippet in debugpy.adapter.NNNN.log:
I+00000.071: Listening for incoming Client connections on 10.40.0.130:51507...
I+00000.071: Listening for incoming Server connections on 127.0.0.1:39415...
I+00000.071: Sending endpoints info to debug server at localhost:60997:
{
"client": {
"host": "10.40.0.130",
"port": 51507
},
"server": {
"host": "127.0.0.1",
"port": 39415
}
}
I+00000.076: Accepted incoming Server connection from 127.0.0.1:43864.
Lastly, I noticed this in debugpy.{adapter,server}.NNNN.log but that seems to be ok, as I also saw this in healthy local runs:
I+00000.049: Error while enumerating installed packages.
Traceback (most recent call last):
File "/my_app/debugpy/adapter/../../debugpy/common/log.py", line 362, in get_environment_description
report(" {0}=={1}\n", pkg.name, pkg.version)
AttributeError: 'PathDistribution' object has no attribute 'name'
Stack where logged:
File "/my_app/python3_x86_64/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/my_app/python3_x86_64/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/my_app/debugpy/adapter/__main__.py", line 227, in <module>
main(_parse_argv(sys.argv))
File "/my_app/debugpy/adapter/__main__.py", line 50, in main
log.describe_environment("debugpy.adapter startup environment:")
File "/my_app/debugpy/adapter/../../debugpy/common/log.py", line 372, in describe_environment
info("{0}", get_environment_description(header))
File "/my_app/debugpy/adapter/../../debugpy/common/log.py", line 364, in get_environment_description
swallow_exception(
File "/my_app/debugpy/adapter/../../debugpy/common/log.py", line 215, in swallow_exception
_exception(format_string, *args, **kwargs)
All of this crashes already before I try connecting to the debugger.
I was also able to reproduce this without using Ray Distributed Debugger. I just connect to the k8s pod, create a small python script:
import debugpy
debugpy.listen(5678)
print("before wait_for_client")
debugpy.wait_for_client()
print("after wait_for_client")
print("before breakpoint")
debugpy.breakpoint()
print("after breakpoint")
Run it and check the log files and see the same crash happening (BrokenPipeError: [Errno 32] Broken pipe) in the pydevd logs.
When I run all of this locally, everything works fine. When running on ray on k8s, I run into this issue...
These are the full, lightly redacted, logs:
Questions:
- Is there a way to detect a crashed listen from code? If so, how?
- Any ideas on what makes this crash?
Expected behavior
Accepting oncoming connections on the debugpy.listen endpoint.
Steps to reproduce:
I'm afraid it will be hard to reproduce this in an environment other than our "ray on k8s" setup. But details are in the "Actual behavior" section.
- Lenguaje dominante
- Python
- Estrellas
- 2.5k
- Forks
- 205
- Merge medio
- 1 d 9 h
- PR fusionados (30 d)
- 3
Preparar el entorno
Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de microsoft/debugpy
-
pydevd_comm.py start_client() does not set TCP_NODELAY on the connections it opensPosiblemente ocupada @nightcityblade la tomó hace 51 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Document "rules"-optionPosiblemente ocupada @nightcityblade la tomó hace 40 días. Abiertodocumentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
-
wx.EventLoop() deprecation warningPosiblemente ocupada @nightcityblade la tomó hace 38 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
debugpy.configure(python=) is not properly documentedPosiblemente ocupada @nightcityblade la tomó hace 49 días. Abiertodocumentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
-
Code run with -c or a "code" launch gets __name__ == "builtins" instead of "__main__"Posiblemente ocupada @Om-singhaI la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 35/100
Todos los issues de microsoft/debugpy
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 83/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 86/100
FuRongJun-1999/dsh-memory#65 ·
Los mantenedores suelen responder en 1 día
-
ci needs-ac
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
Ikalus1988/MisakaNet#2930 ·
Los mantenedores suelen responder en 1 día
-
`FakeBackendV2.run` fails with `NoiseError` on circuits with delays on qubits where T2 > 2·T1Abiertobug
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Qiskit/qiskit-aer#2466 ·
-
area/cli
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100