device-mapper-utils/blk-stash

Provide way to rebuild offsets file for slab files

Open

#18 创建于 2025年7月23日

在 GitHub 查看
 (0 评论) (0 反应) (0 负责人)Rust (1 fork)auto 404
good first issuehelp wanted

仓库指标

Star
 (0 star)
PR 合并指标
 (PR 指标待抓取)

描述

Provide way to rebuild offsets file for slab files; compare timestamps and trigger automatically

Summary

The offsets file associated with each slab file is critical for efficiently locating and accessing chunk data. However, if the offsets file is lost, corrupted, or out-of-sync with its corresponding slab, we currently have no automated way to detect or regenerate it.

We should add support to:

  1. Automatically detect stale or missing offsets files by comparing file timestamps or size/checksum checks
  2. Rebuild the offsets file from the corresponding slab if necessary
  3. Integrate this logic into archive validation, verification, or startup routines

Goals

  • Ensure archives remain usable even if offsets files are missing or invalid
  • Prevent silent errors or performance degradation due to desynchronized slab/index pairs
  • Improve robustness and self-healing capabilities

Tasks

  • Define a method to compare timestamps (e.g., slab mtime > offsets mtime)
  • Add logic to scan slab files and regenerate offsets file when:
    • Offsets file is missing
    • Offsets file is older than the slab
    • Offsets file is malformed or unreadable or fails checksum checks
  • Add logging to indicate when an automatic rebuild occurs
  • Add a CLI command (e.g., blk-archive rebuild-offsets) for manual recovery
  • Include checks in verify and/or restore paths to trigger rebuilds as needed
  • Write test cases covering scenarios:
    • Missing offsets
    • Out-of-date offsets
    • Corrupt offsets
  • Document behavior and limitations

Considerations

  • Avoid unnecessary rebuilds when offsets are valid
  • Warn the user if rebuild fails or results in mismatched metadata
  • Ensure performance is acceptable for large slabs during scanning
  • Could optionally store a checksum or slab UUID in offsets for validation

贡献者指南