device-mapper-utils/blk-stash
在 GitHub 查看Add `migrate` subcommand to move streams between archives (essential for garbage collection)
Open
#17 创建于 2025年7月23日
help wanted
仓库指标
- Star
- (0 star)
- PR 合并指标
- (PR 指标待抓取)
描述
Add migrate subcommand to move streams between archives (essential for garbage collection)
Summary
Currently, blk-archive does not support deleting individual streams from an archive, which limits our ability to reclaim space through traditional garbage collection. This is due to the fact that slab files are append only and are never edited in place. To work around this, we need a migrate subcommand that copies selected streams from one archive to another—leaving behind unreferenced or obsolete data.
This capability is essential for long-term archive maintenance, compaction, and reclaiming space when references are dropped.
Goals
- Implement a
blk-archive migratesubcommand that:- Selectively copies streams and associated data from a source archive
- Writes the selected streams into a new or existing destination archive
- Avoids unnecessary duplication of already-present slabs or chunks
- Support migration of only live/in-use references
- Provide tooling to assist in archive compaction and reorganization
Tasks
- Define CLI and UX for the
migratesubcommand (e.g., input/output paths, filters) - Walk and copy only reachable slabs, chunks, and metadata
- Avoid re-archiving already-present content in the destination archive
- Validate consistency after migration
- Add tests for partial and full migrations
- Document use cases (e.g., archive shrinking, repacking)
Use Cases
- Garbage collection by "copying live data forward"
- Archival rotation or tiered storage
- Recovering from damaged or bloated archives
Considerations
- May require metadata enhancements to track reference counts or root sets more explicitly
- Must maintain slab compatibility and integrity across archives
- Could be a foundation for future GC or snapshot pruning tools