Make gc() reachable so orphaned blobs and manifests are actually reclaimed
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 45/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 静か
- 技術スタック
- sqlite, typescript
調査の方向性
packages/dofs/src/fs/gc.ts と packages/dofs/src/index.ts から始め、次に packages/computer/src/workspace.ts の Workspace エントリーポイントと既存の gc テストを調査します。クリーンアップが呼び出し元主導なのか、アラームベースなのか、または機会的なのかを明らかにし、選択したエントリーポイントと結果を公開し、孤立した行が回収される一方でリンクされた blob は変更されないように、4 つのコメントとテストを更新します。
索引モデルが issue の本文から書いたものです。
説明
Summary
packages/dofs/src/fs/gc.ts implements orphan reclamation for vfs_blobs and vfs_manifests. It is transactional, it has a one-hour safety window, it has unit tests, and the schema carries a partial index added specifically to keep its manifest sweep from being quadratic. It is not exported and nothing calls it, so no orphan is ever reclaimed.
Filing as an issue because CONTRIBUTING.md routes feature requests to Discussions and Discussions are not enabled on this repo, so the documented link 404s (#53). Happy to move this to a Discussion if that gets turned on.
Background and motivation
Verified against main at 76d9e75:
gcis not exported frompackages/dofs/src/index.ts. That file exports 30-plus symbols including every sync building block (applyChanges,coalesceChanges,fetchChanges,stageBlob,writeWatermark,buildManifest).gcis absent.packages/dofs/package.jsonexposes only"."and"./testing", so there is no deep import path either.- Searching for
gc(acrosspackages, excludingfs/gc.tsand tests, returns four hits, and all four are comments.
Those four comments are the reason this matters. Three of them are load-bearing justifications for leaving garbage behind:
packages/dofs/src/fs/writeFile.ts:138- "Failure mid-stream leaves blob rows behind;gc()reaps orphans on a later pass."packages/dofs/src/fs/writeFile.ts:152- "orphan blob rows thatgc()then has to reap."packages/dofs/src/fs/writeFile.ts:961- "are cleaned up by a latergc()pass."packages/computer/src/mounts/types.ts:26- "rows may briefly linger and are reaped bygc()."
There is no later pass. Every interrupted or failed streaming write leaks blob rows, and the bytes go with them through the vfs_blob_bytes foreign key. In a Durable Object, where SQLite is the durable substrate and storage is finite, a cleanup that is documented but never runs is a slow leak with no operator remedy short of recreating the workspace.
The surrounding evidence suggests this is an oversight rather than a decision:
gc.ts:20-53runs both deletes in onedb.transactionSync, withDEFAULT_SAFETY_WINDOW_MSof one hour and the comment that the generous default exists so "a misconfigured GC pass cannot wipe blobs the application is actively writing." That is operational thinking, not dead code.packages/dofs/src/schema/core.ts:51-57adds the partial indexvfs_nodes_by_manifest_hashwith the comment: "gc/manifests checks every manifest row against vfs_nodes via a correlated NOT EXISTS [...] Without this index gc full-scans vfs_nodes per candidate manifest - O(N x M)." Schema work was done for a function that cannot run.packages/dofs/src/sync/blobs.ts:10hasstageBlobtouchlast_seen"so the bytes don't get reaped by an interleaved gc," so the sync path already coordinates with it.packages/dofs/README.mdnotes thesrc/fs/*primitives,gcamong them, "are not re-exported from the package root yet."
Goals
-
Make
gcreachable: exportgc,GcOptions, andGcResultfrom the@cloudflare/dofspackage root, alongside the sync building blocks already exported there. No behaviour change. -
Give it a host-side entry point on
Workspacethat runs the sweep and returns{ blobsFreed, manifestsFreed }, so a consumer can reclaim on its own schedule and observe what was freed. -
Decide who triggers it. This is the part I would rather ask than assume, and it is the part that actually closes the leak. Three options with different cost profiles:
- Caller-driven only. Smallest change, no policy baked in, but the leak persists for every consumer who does not know to call it.
- A Durable Object alarm. The natural home for periodic maintenance, but
packages/computer/src/workspace.ts:57notes a backend "cannot own a Durable Object alarm. Each backend has at most one intent," and the container keep-alive already uses alarms, so the alarm is contended. - Opportunistic, after a sync tick or the post-exec pull, guarded by the existing one-hour
last_seenwindow plus a cheap "anything deleted since last sweep" check so an idle workspace does no work. Cheapest to reason about, but adds work to a latency-sensitive path.
The safety window means correctness does not depend much on the choice; cost and latency do.
-
Make the four comments true. If the answer is caller-driven only, they should be amended to say the caller is responsible, so the code stops asserting a cleanup the library does not perform.
Out of scope: no change to gc's predicates, safety window, or transaction shape. It looks correct as written; it is only unreachable. Tombstone pruning in vfs_changes is a separate gap I am filing alongside this one.
Deleting gc and its index instead is a coherent alternative if orphans are considered acceptable, and worth naming so the decision is explicit. It would mean rewriting the four comments and accepting the leak from interrupted writes.
Example
// packages/dofs/src/index.ts - currently absent
export { gc, type GcOptions, type GcResult } from "./fs/gc.js";
// host-side entry point
const { blobsFreed, manifestsFreed } = await workspace.gc();
For tests, the now injection already in GcOptions exists so the clock can be pinned, so no new test infrastructure is needed. The coverage worth adding is the reachability regression that would have caught the current state (importing gc from the package root and running it), plus driving an interrupted streaming write through the writeFile.ts:138 path, asserting orphan rows exist, sweeping past the safety window, and asserting they are gone while linked blobs are untouched.
Happy to open a PR for the export and the entry point if that direction works, and to hold the trigger question until you have picked one.
- 主要言語
- TypeScript
- スター
- 9.2k
- フォーク
- 525
- 平均マージ
- 3日 12時間
- マージ済み PR(30日)
- 18
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
cloudflare/computer のほかの issue
-
enhancement
cloudflare/computer#129 · 担当者 1 名 ·
-
bug
難易度 3/5 1〜2日 初心者へのやさしさ 65/100
cloudflare/computer#106 · コメント 3 件 ·
-
難易度 5/5 1週間以上 初心者へのやさしさ 45/100
cloudflare/computer#67 · コメント 1 件 ·
-
bug
難易度 4/5 3〜5日 初心者へのやさしさ 48/100
cloudflare/computer#52 · コメント 3 件 ·
-
enhancement
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
cloudflare/computer#50 · コメント 5 件 ·
cloudflare/computer の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 84/100
receptron/mulmoterminal#2264 ·
-
documentation
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
components-web-app/docs#96 ·
-
enhancement
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
simonsobs/tileviewer#114 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100