v6.3.0 introduced pkg_resources side effects

Abierto

#824 abierto el 14/12/2021 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad: 4/5
Tiempo estimado: 3-5 días
Aptitud para principiantes: 35/100

Tipo de issue: Error
Claridad: Bastante claro
Estado de actividad: Estancado
Stack tecnológico: jupyter-notebook, python
Área: backend

Línea de trabajo

Comienza con el commit 470145ca7d103bdf6081bcfde09933ab81a56a5b y sigue la importación a través de debugpy.server.api hasta debugpy/_vendored/pydevd/pydevd_plugins/init.py. Ejecuta el notebook de reproducción enlazado con ipykernel<6.3.0, ipykernel==6.3.0 y el head actual, observando sys.path y pkg_resources. Se considera terminado cuando el reproductor ya no falla sin una solución específica para notebooks y se entiende cualquier impacto en la depuración de pydevd.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug

ipykernel v6.3.0, I believe 470145ca7d103bdf6081bcfde09933ab81a56a5b specifically, introduced an indirect import on the pkg_resources module, via debugpy.server.api.

During this import, debugpy imports pkg_resources (here).

The pkg_resources module has some import-time side-effects that cause problems in complex notebooks:

It’s important to note that the global working_set object is initialized from sys.path when pkg_resources is first imported, but is only updated if you do all future sys.path manipulation via pkg_resources APIs.

I am filing this with ipykernel because I believe it's the place we start caring about the sys.path manipulations - it is somewhat common for notebooks to include !pip install foo steps, which impacts sys.path without using the pkg_resources APIs. However, I'd like to discuss where a fix should live.

I'm part of the TensorFlow docs team, and we have an increasing number of notebooks that have a "restart runtime" requirement (e.g.). I don't think fixing every notebook is the right solution, and we could apply a fix in our own infrastructure, but I would favour a fix at the root, if possible.

When we fix it in the notebook, we reload the module (e.g.), however we should unload the module completely if we fix this before any notebook code, in case the notebook uses pkg_resources. If we were to do this, say, in ipykernel, would it impact the notebook kernel or other environment factors? e.g. in the use of pydevd or debugging?

In case it helps, this notebook can be used to reproduce the failure. ipykernel<6.3.0 works, but ipykernel==6.3.0 (and at head) fail.