labs install: stale virtualenv with a dangling interpreter symlink is never repaired, so install fails permanently
Los mantenedores suelen responder en 1 día
Evaluación
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 68/100
Línea de trabajo
Empieza por el flujo de instalación de labs que crea el directorio state/venv y después invoca su bin/python3 para instalar las dependencias. Reproduce el caso de un symlink obsoleto colgante, inspecciona cómo se gestionan la creación de la venv y los errores de pip, y añade o actualiza la cobertura si se encuentran las pruebas pertinentes. Se considera terminado cuando una venv rota reutilizada se reconstruye o el error identifica el intérprete que falta, y los detalles de pip o exec dejan de ocultarse.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Describe the issue
databricks labs install <project> reuses the existing virtualenv directory at ~/.databricks/labs/<project>/state/venv by running python -m venv <dir> over it.
Python's venv module will not replace files or symlinks that already exist in the target directory unless --clear is passed. If the venv's bin/python3 symlink has become dangling — which happens routinely on macOS whenever Homebrew retires the Python minor version the venv was originally built against — then:
python -m venv <dir>reports success and rewritespyvenv.cfgto advertise the newly detected interpreter version.bin/pythonandbin/python3still point at the now-deleted old interpreter.- The installer then invokes
<dir>/bin/python3 -m pip install ..., which fails instantly because that path does not exist. - The command exits with
Error: python: failed to install dependencies of .
Because nothing in the flow ever repairs or discards the broken venv, every subsequent retry fails identically. The install is unrecoverable without manually deleting the venv, and nothing in the output hints that this is the problem.
Steps to reproduce the behavior
# 1. Install a labs project while [email protected] is the newest Homebrew Python
databricks labs install lakebridge
# 2. Let Homebrew retire that interpreter (any of these will do it over time)
brew upgrade
brew cleanup
# ~/.databricks/labs/lakebridge/state/venv/bin/python3 is now a dangling symlink
# 3. Re-run the install
databricks labs install lakebridge
# Error: python: failed to install dependencies of .
# ...and it will fail this way forever
Confirming the broken state:
$ ls -la ~/.databricks/labs/lakebridge/state/venv/bin/python3
python3 -> python3.13 -> /opt/homebrew/opt/[email protected]/bin/python3.13 # does not exist
$ cat ~/.databricks/labs/lakebridge/state/venv/pyvenv.cfg
home = /opt/homebrew/opt/[email protected]/bin
include-system-site-packages = false
version = 3.10.20 # <-- claims 3.10.20, but bin/python3 still points at 3.13
Expected Behavior
labs install should end up with a working virtualenv, or fail with an error that says what is actually wrong. Any of the following would resolve it:
- Create the venv with
--clear(orEnvBuilder(clear=True)), so a reused directory is rebuilt from scratch. os.RemoveAllthe venv directory before creating it — this is an install command, a clean environment is the reasonable expectation.- Validate that
<venv>/bin/python3resolves to an existing, executable file after creation, and rebuild if it does not.
Actual Behavior
Install fails with Error: python: failed to install dependencies of . and every retry fails identically until the stale venv is deleted manually.
Second, smaller issue: pip's stderr is swallowed
The failure surfaces only as:
Error: python: failed to install dependencies of .
The underlying error was no such file or directory: .../state/venv/bin/python3. Propagating pip's actual stderr (or even just the exec error) would turn this from a debug-log archaeology exercise into an immediately obvious fix. Happy to split this into a separate issue if preferred.
Why this is worth fixing
This is not project-specific — it affects every labs project (ucx, lakebridge, and others), since they all share the same install path. On macOS with Homebrew it is essentially a matter of time: any brew upgrade or brew cleanup that removes an old [email protected] formula silently breaks every venv built against it, and the labs installer has no way to recover.
OS and CLI version
- Databricks CLI: v1.3.0
- Labs project:
lakebridgev0.14.2 (requires-python = ">=3.10.1,<3.15") - OS: macOS 26.5.2 (arm64)
- Pythons present: Homebrew 3.10.20 and 3.14.6, system 3.9.6
- Previously present (now removed by Homebrew): 3.13
Is this a regression?
Unknown — observed on CLI v1.3.0 after a Homebrew Python upgrade removed the interpreter the existing labs venv was linked to.
Debug Logs
15:39:24 Info: start pid=NNNNN version=1.3.0 args="databricks, labs, install, lakebridge, --profile=REDACTED, --log-level=DEBUG"
15:39:25 Debug: Latest lakebridge version is: v0.14.2 pid=NNNNN
15:39:37 Debug: Unpacking zipball to: ~/.databricks/labs/lakebridge/lib pid=NNNNN
15:39:38 Debug: found 8 potential alternative Python versions in $PATH pid=NNNNN
15:39:38 Debug: /opt/homebrew/.../python3.14 --version: Python 3.14.6 pid=NNNNN
15:39:38 Debug: /opt/homebrew/.../python3.10 --version: Python 3.10.20 pid=NNNNN
15:39:38 Debug: /usr/bin/python3 --version: Python 3.9.6 pid=NNNNN
15:39:39 Debug: Detected Python v3.10.20 at: /opt/homebrew/Cellar/[email protected]/3.10.20_4/Frameworks/Python.framework/Versions/3.10/bin/python3.10 pid=NNNNN
15:39:39 Debug: Creating Python Virtual Environment at: ~/.databricks/labs/lakebridge/state/venv pid=NNNNN
15:39:39 Debug: running: /opt/homebrew/.../python3.10 -m venv ~/.databricks/labs/lakebridge/state/venv pid=NNNNN
15:39:41 Debug: Installing Python dependencies for: ~/.databricks/labs/lakebridge/lib pid=NNNNN
15:39:41 Debug: running: ~/.databricks/labs/lakebridge/state/venv/bin/python3 -m pip install --upgrade --upgrade-strategy eager . pid=NNNNN
Error: python: failed to install dependencies of .
15:39:41 Info: failed execution pid=NNNNN exit_code=1 error="python: failed to install dependencies of ." pid=NNNNN
Workaround
rm -rf ~/.databricks/labs/<project>/state/venv
databricks labs install <project>
- Lenguaje dominante
- Go
- Estrellas
- 404
- Forks
- 246
- Merge medio
- 1 d 11 h
- PR fusionados (30 d)
- 229
Preparar el entorno
- Incluye un Dockerfile o un archivo de Docker Compose
- Tiene una plantilla de pull request
- Sin guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de databricks/cli
-
Apps commands only accept relative path to app.ymlPosiblemente ocupada @yuri-rod la tomó hace 9 días. AbiertoCLI
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
databricks/cli#6910 ·
Los mantenedores suelen responder en 1 día
-
DABs
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
databricks/cli#6670 ·
Los mantenedores suelen responder en 1 día
-
DABs
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
databricks/cli#7032 ·
Los mantenedores suelen responder en 1 día
-
DABs engine/direct Enhancement
Dificultad 3/5 1-2 días Aptitud para principiantes 56/100
databricks/cli#7013 · 2 reacciones ·
Los mantenedores suelen responder en 1 día
-
DABs
Dificultad 5/5 Más de una semana Aptitud para principiantes 18/100
databricks/cli#7002 ·
Los mantenedores suelen responder en 1 día
Todos los issues de databricks/cli
Issues similares
-
state:triage-needed
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
puppe1990/amarra-cais#342 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
Los mantenedores suelen responder en 1 día
-
tier:1-mechanical
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
krakenhavoc/cmd_and_ctrl#3278 ·
Los mantenedores suelen responder en 1 día