Add fields to slab file header to describe its contents and format version

Add fields to slab file header to describe its contents and format version

Summary

The current slab file format lacks explicit metadata in its header to describe the contents and version of the format. Adding structured fields to the header would improve forward compatibility, format introspection, and error checking.

Goals

• Introduce a clear version identifier for the slab file format
• Optionally include if missing descriptive fields (e.g., slab type, compression method, checksum algorithm)
• Enable tools to reject incompatible or unknown slab formats early
• Aid is reconstruction damaged archive
• Aid debugging, future format upgrades, and backward compatibility logic

Tasks

• Define the structure and layout of the header extension
• Add a version field (e.g., u32, magic value + semantic version?)
• Add optional fields for:
  • Compression method (if any)
  • Hashing/checksum algorithm
  • Slab type or role (e.g., data, index, metadata)
  • Reserved space for future flags or extensions
• Update slab creation code to write the new header format
• Update slab parsing code to detect and parse the header
• Add validation logic for version compatibility
• Add unit tests for legacy vs. updated formats
• Document the format in doc/ or in-source comments

Considerations

• Choose a layout that allows easy expansion without breaking compatibility (e.g., TLV, offset-based)
• Ensure we're using a unique magic numbers where appropriate to help distinguish different file types from arbitrary data
• Ensure old readers either reject unknown headers cleanly or fall back safely
• Compose test to ensure compatibility between versions/changes

Benefits

• Enables format evolution without breaking tools
• Makes the slab file format self-describing
• Improves robustness and debuggability
• Allows the ability to potentially heal damage to archive