pypa/pipx

space efficient non-isolated caches? or use deps solver like `micromamba`?

Open

#1,254 opened on Feb 9, 2024

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Python (7,844 stars) (363 forks)batch import
enhancementhelp wanted

Description

Discussed in https://github.com/pypa/pipx/discussions/1167

Originally posted by goyalyashpal December 24, 2023 hi all! greetings of the day.

i just found pipx & was thinking to use it for things like:

  • ides:thonny, jupyterlab,
  • python devtools: ruff black mypy autopep8 flake8
  • utils like: pygments
  • ... etc

currently,

  • other than few shared/Lib like pip, setuptools
  • rest all go to their own isolated dirs (right?)
  • which will increase the duplicacy over time and consume lots of space
  • like i want to use pygments as an app for myself, but other python apps like jupyter use it as a lib too, so, unnecessary duplication for isolation is just plain wastage of resources

so, i was thinking if it would be possible to:

  • store atomic & versioned package in one common cache dir
  • and then link the required libs from there using symlinks etc...

this seems to be similar to the way

  • the proper dependency solvers like micromamba do it in python world,
  • or the pnpm does for the node world.
  • or the nix does for the system-level packages and configurations

Similar issues elsewhere:

Contributor guide

Tech stack
python
Domain
tooling
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
needs maintainer response
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
pipx internalsPython virtual environmentsdependency resolutionsymlinks
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
15
Research direction
Investigate feasibility of implementing a shared cache for pipx, similar to pnpm or micromamba. Examine pipx's current caching in `pipx/commands/` and `pipx/shared.py`. Consider challenges of cross platform symlinks and dependency resolution. Evaluate `micromamba` as a potential integration model.