LayerFS Paper Write in Public

This is a tracking issue for a systems paper, not a single edit. Start with docs/research/paper/OUTLINE.md for the gap list and layerfs.tex for the red-marked gaps, then check the SQLite directory-lock claim in the "Parallel objects, single-writer metadata" section against the layerfs_v2_0_0 source. Done means the checklist items are ticked, each number is traced to a benchmark report or commit, and the draft is merged into main.

Write the LayerFS systems paper in public: draft, review and revise it in this repository, so every claim can be checked against the code and the published benchmark data.

Working title: LayerFS: A Recursive Filesystem for Ephemeral Computing in AI Agents

Files

The draft is on branch paper/write-in-public.

Main paper

• layerfs.pdf, the built draft. Gaps are marked in red.
• layerfs.tex and references.bib, the source.
• Appendices:
  • appendix.tex: architecture, committed state, overlay and algorithms;
  • appendix-cli.tex: all 73 CLI calls;
  • appendix-pipeline.tex;
  • appendix-analysis.tex.
• README.md, the index and build command, and OUTLINE.md, the outline, evaluation plan and gap list.

Supporting research notes

• A: website claims and core ideas
• B: engine design
• C: core storage (CAS, CDC, deltas, packs)
• D: speed history
• E: evaluation tables and the agent-time premise
• Appendix notes:
  • algorithms
  • modal comparison
  • overlay
  • pipeline
  • architecture
  • structures
• CLI reference (73 calls)
• Premise scripts: read-only, and they print aggregates only.

Links

• Core ideas: https://layerfs.ai/docs/core-ideas
• Code referenced by the paper: layerfs_v2_0_0 at 8cfb6e2734
• Benchmark suites and released data: layerfs-benchmark

Thesis

Agents spend most of a session thinking, not running commands, and many of their calls are file reads and edits that need no computer. Because of this, the unit of execution should be a cheap, short-lived Workspace over shared, immutable history, not a long-lived machine per agent. LayerFS records the whole filesystem state, including ignored files and installed dependencies, so nothing is reinstalled. It can roll out from any saved state, fork recursively without copying, and keep storage close to the size of what actually changed.

What the paper covers

• Ephemeral Workspaces. File calls run against the Store without a Sandbox. A mount costs the same whatever the tree size. Each Workspace has its own overlay. Commit cost follows the size of the change, and reclamation follows unmount.
• Shared history. A fork is one Branch row, made at any depth and from any Commit or Layer. Several agents can work on one Branch. Merge-on-commit is three-way. Search runs over filesystem states.
• Single writer for metadata, many writers for objects. Pack bodies are written in parallel. The locator index and history are published through one short publication lane.
• Recording everything: share first, then compress.
  • BLAKE3 object identity, structural sharing and exact reuse.
  • Content-defined chunking and localized edits.
  • Delta bases chosen from filesystem events (rename-over, recreate-at-path), with a similarity fallback.
  • Bounded delta chains, metadata programs and packs.
• The interface. One call table generates the CLI and the Node and Python SDKs: 73 calls in ten families. The full CLI is in an appendix.
• Evaluation.
  • A 13-hour, nine-agent build.
  • Storage compared with agent and versioned filesystems (AgentFS, ovnfs, btrfs, overlayfs).
  • Real repository histories, the effect of commit granularity and the contribution of each mechanism.
  • Per-call latency, and where LayerFS is slower.

Ground rules

• Experiments use v1-era builds and published layerfs-benchmark reports. Each table names its build. No reruns are planned for the paper.
• Design and code references point at layerfs_v2_0_0.
• The comparisons are agent and versioned filesystems, not Git.
• Weaknesses are reported:
  • read throughput;
  • import is 6–8× slower;
  • some new-file workloads are slower;
  • no GC yet;
  • only the Disposable durability profile.
• Session-analysis numbers are aggregates only. Raw transcripts are not published.

Checklist

• Draft, PDF and research notes on paper/write-in-public
• Merge the draft into main under docs/research/paper/
• Fix the known inaccuracy in "Parallel objects, single-writer metadata": SQLite object writes take the directory lock and can wait while the lane is held
• Close the gaps marked in red in the draft
• Review each section against the layerfs_v2_0_0 source
• Check every number against its benchmark report or commit
• Related work: final comparison with AgentFS, ovnfs, overlayfs, btrfs and the dedup/delta literature
• Choose a venue and format
• Publish a preprint