layerfs rm/compact: retire Branches and reclaim unreachable Store history

未关闭
  • 哪里不对？
#411 7 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

@SSK015 已经在做这个了。

开始于 2026年10月27日。

评估

难度
5/5
预计耗时
一周以上
新手友好度
25/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
rust, sqlite
领域
backend, databases

调研方向

Start with core/docs/architecture/84-global-sqlite-store.md §8 and branch.md, which the issue cites to describe current Store and Branch behavior. The issue asks for a safe contract to retire Branches and reclaim unreachable history, but leaves the design and safety rules open. Done would mean a defined approach for both operations that accounts for objects reachable from Branch heads, Layers, and open Workspaces.

由索引模型根据 Issue 内容生成。

描述

new feature

Summary

A project's Store has no way to give space back. Every Branch, Commit and Layer stays, including committed build output, and the SDK has no call to retire a Branch (branch.md lists fork, list_commits, diff_commits, diff_branches, merge, publish). core/docs/architecture/84-global-sqlite-store.md §8 says so: "Safe garbage collection of unreachable history needs its own contract and is not implemented"; free pages are reclaimed, unreachable history is not. I found no open issue tracking it, so this is one, with what we see in real use.

Why it matters for agent work

The agent presets ask agents to commit after every command, "sources, lockfiles, node_modules, build output", and to merge generated paths newer-wins through /.lww. That is what makes forks warm, and it means each rebuild that changes target/ or node_modules adds its new bytes to history for good. Agents also make many short-lived Branches (one per task, plus scratch and verification Branches) that are never merged or published. Over weeks the only way to shrink a project is to start a new one.

Numbers from two real projects (LayerFS 5fb68dd, Linux, WSL2)

Project Tree Store
tdxcli-rs (Rust workspace, 485 crates) project.init of a tree with a built target/: 5.05 GB 1.70 GB after init; 3 agents' Commits after a one-line edit and incremental rebuild added 0.11 GB
KATANAview (pnpm monorepo; node_modules 1,004 MB, 57,931 files) 18 Commits, 8 Branches, 2 Layers; 28.5 GB if every Commit's snapshot were stored raw 722 MB on disk per the host's storage read (GET /layerfs/project/<p>/storage, DSH's Storage figures); the Store volume is 1.1 GB (du)

Dedup and compression keep each step small, which is the good news. The cost is that none of it can be reclaimed: 4 of those 8 KATANAview Branches are finished smoke, baseline and verification Branches, and a cargo clean, a toolchain bump or a lockfile change adds a new generation of target/ / node_modules that stays even after every Branch holding it is abandoned.

What we do today

Our integration rotates a project: export main's files, project.init a new project from them, rebind, delete the old Store volume. It works, but it loses all History and every unmerged Branch, so it is a last resort.

Question

Is there a planned shape for this contract? Two pieces would cover agent use:

  1. retiring a Branch (it leaves the namespace; its Commits become unreachable unless another Branch or a Layer holds them);
  2. reclaiming objects no Branch head, Layer or open Workspace reaches, explicitly and bounded like the existing free-page reclamation.

Happy to test a proposal on these projects and report Store sizes before and after.

主要语言
Rust
星标
2.1k
派生
110
平均合并
23 小时 52 分钟
30 天内合并 PR
138

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

Ephemeral-AI-Lab/layerfs 的其他 Issue

查看 Ephemeral-AI-Lab/layerfs 的全部 Issue

相似的 Issue

更多 Rust Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。