device-mapper-utils/blk-stash

Add `migrate` subcommand to move streams between archives (essential for garbage collection)

Open

#17 创建于 2025年7月23日

在 GitHub 查看
 (0 评论) (0 反应) (0 负责人)Rust (1 fork)auto 404
help wanted

仓库指标

Star
 (0 star)
PR 合并指标
 (PR 指标待抓取)

描述

Add migrate subcommand to move streams between archives (essential for garbage collection)

Summary

Currently, blk-archive does not support deleting individual streams from an archive, which limits our ability to reclaim space through traditional garbage collection. This is due to the fact that slab files are append only and are never edited in place. To work around this, we need a migrate subcommand that copies selected streams from one archive to another—leaving behind unreferenced or obsolete data.

This capability is essential for long-term archive maintenance, compaction, and reclaiming space when references are dropped.

Goals

  • Implement a blk-archive migrate subcommand that:
    • Selectively copies streams and associated data from a source archive
    • Writes the selected streams into a new or existing destination archive
    • Avoids unnecessary duplication of already-present slabs or chunks
  • Support migration of only live/in-use references
  • Provide tooling to assist in archive compaction and reorganization

Tasks

  • Define CLI and UX for the migrate subcommand (e.g., input/output paths, filters)
  • Walk and copy only reachable slabs, chunks, and metadata
  • Avoid re-archiving already-present content in the destination archive
  • Validate consistency after migration
  • Add tests for partial and full migrations
  • Document use cases (e.g., archive shrinking, repacking)

Use Cases

  • Garbage collection by "copying live data forward"
  • Archival rotation or tiered storage
  • Recovering from damaged or bloated archives

Considerations

  • May require metadata enhancements to track reference counts or root sets more explicitly
  • Must maintain slab compatibility and integrity across archives
  • Could be a foundation for future GC or snapshot pruning tools

贡献者指南