Bundle cache grows unbounded — no dep dedup, no eviction policy
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 39/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- python, typescript
Research direction
Start with packages/sdk-typescript/src/cache.ts, then read prepareLocalBundle() in packages/sdk-typescript/src/mpakSDK.ts and hashBundlePath in packages/sdk-typescript/src/helpers.ts. The issue recommends a phased cache retention and eviction policy, but leaves configuration and CLI behavior to design. Done means phase 1 meets the listed retention, size cap, info, prune, and under-cap behavior criteria.
Written by the indexing model from the issue text.
Description
Problem
The bundle cache at ~/.mpak/cache/ accumulates without bound and duplicates vendored Python dependencies across every cached bundle. Real-world example from one developer's machine after a few months of use:
- Total cache: 3.7 GB
_local/: 2.1 GB across 33 entries (local.mcpbinstalls frommpak install <path>)- 16 of those entries are 76 MB each and represent the same bundle rebuilt during dev iteration
- Each entry's
deps/dominates: pandas (48 MB), numpy (23 MB), cryptography (22 MB), pydantic, fastmcp, etc., copied per-bundle
Two distinct contributors:
- Local-rebuild churn.
prepareLocalBundle()keys cache entries by a hash of the bundle's absolute path (packages/sdk-typescript/src/mpakSDK.ts:280). A versioned filename likedist/mcp-foo-v0.1.0.mcpb→v0.1.1.mcpbchanges the path, so each rebuild produces a new cache entry; the old one is never reclaimed. Dominant cause of the 2.1 GB. - Cross-bundle duplication. Every bundle's
deps/is extracted as its own copy. Pandas/numpy/cryptography exist N times for N bundles, even though the wheels are usually byte-identical. Grows linearly with bundle count.
There is no eviction policy, no size cap, and no GC. The only cleanup path is manual rm -rf.
Options considered
| Approach | Fixes churn | Fixes cross-bundle dup | Cost | Notes |
|---|---|---|---|---|
| A. Retention policy + size cap | yes | no | trivial (~50 lines) | Per-localPath keep-latest-N in _local/; global LRU/size cap across cacheHome. Stopgap. |
B. APFS clonefile / Linux reflinks during extract |
yes | yes (partial) | small | Identical file contents share blocks at the FS layer. Needs a content-hash index to detect "seen this file before." macOS-only natively (APFS); Linux needs btrfs/xfs reflinks. |
| C. Shared venv (one env, all bundles) | yes | yes | medium | Disqualified — version conflicts across bundles are exactly why deps are vendored. |
| D. Content-addressable storage + hardlinks | yes | yes | significant rewrite | General, correct, prior art (pnpm, Nix, Bazel). Hardlink edge cases (__pycache__, cross-FS). |
E. Delegate to uv (lockfile in bundle, uv resolves on install) |
yes | yes | medium | Changes bundle contract; requires online install or pre-warmed wheel cache. uv already implements CAS+hardlinks for wheels. |
| F. Symlink farms | yes | yes | medium | Python + symlinks + __pycache__ is a known footgun. |
| G. Overlay/FUSE | yes | yes | high | Way too heavy for a single-machine disk cache. |
Recommendation
Phased:
-
Ship A first (low-risk, kills the dominant cause):
- In
_local/, retain only the most recent extraction perlocalPathfrom.mpak-local-meta.json. Evict older entries. - Add an opt-in size cap on
cacheHomewith LRU eviction across both_local/and named registry caches. - Expose
mpak cache pruneandmpak cache info.
- In
-
Then evaluate B (FS clones) for cross-bundle dedup. Lowest-effort path to dedup that doesn't require restructuring storage. macOS gets it via
clonefile(2); Linux viaFICLONEioctl on btrfs/xfs; ext4 falls back to regular copy. -
Reach for D or E only if data justifies it. mpak is a single-machine, single-user disk cache — it does not need the generality CAS protocols provide for multi-host or build-system contexts. Delegating to uv (E) is attractive but changes the bundle contract and deserves its own design discussion.
Files
packages/sdk-typescript/src/cache.ts—MpakBundleCache, registry-bundle cache layoutpackages/sdk-typescript/src/mpakSDK.ts:275—prepareLocalBundle,_local/<hash>entry creationpackages/sdk-typescript/src/helpers.ts:267—hashBundlePath(path-keyed, not content-keyed)
Acceptance for phase 1
-
_local/retains at most N entries perlocalPath(config, default 1) - Cache total size capped via LRU eviction (config, default off or generous)
-
mpak cache inforeports total size, per-bundle size, oldest entry -
mpak cache pruneruns the eviction passes manually - No behavior change when cache is small / under cap
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 13
- Avg merge
- 16m
- Merged PRs (30d)
- 4
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NimbleBrainInc/mpak
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
NimbleBrainInc/mpak#180 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NimbleBrainInc/mpak#175 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
NimbleBrainInc/mpak#159 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
NimbleBrainInc/mpak#120 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
NimbleBrainInc/mpak#112 ·
All issues in NimbleBrainInc/mpak
Similar issues
-
bug documentation
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Potential Bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Comfy-Org/ComfyUI_frontend#23773 ·
Maintainers usually reply within 1 day
-
community first-timers-only good first issue hacktoberfest help wanted low hanging fruit up-for-grabs
Difficulty 1/5 Under an hour Newbie friendliness 85/100
lingdojo/kana-dojo#34301 · 1 comment · 5 reactions ·
Maintainers usually reply within 1 day
-
code-quality refactoring
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
github/gh-aw-firewall#11797 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
code-yeongyu/senpi#7007 ·
Maintainers usually reply within 1 day