Transfer file blocks instead of the whole file

[!CAUTION] Do not work on this issue yet.
This is intended only for the first Beta Release and must not be started before then.

Currently, all files are handled as a single unit. When a change occurs in a file, the entire file is transferred to other peers, even if only a small part has changed. This is inefficient for large files or for frequent small edits.

Goal

Implement a mechanism to split each file into chunks/blocks, and when a change is detected in a specific block, transfer only that block to other peers instead of the whole file.

High-Level Requirements

• Introduce a concept of file blocks/chunks:
  • Define how files are split (fixed-size blocks, content-defined chunks, etc.).
  • Ensure the chunking is deterministic so that all peers can agree on block boundaries.
• Detect which blocks have changed when a file is modified.
• Transfer only the changed blocks (plus any necessary metadata) to other peers.
• Ensure data integrity:
  • Consider using checksums/hashes per block.
  • Validate blocks upon receipt.
• Maintain compatibility with the existing protocol:
  • Define a migration or fallback path for peers that only support whole-file transfers.
  • Ensure that older peers can still synchronize correctly (even if less efficiently).

Considerations / Open Questions

• Chunk size strategy: fixed size vs. variable (e.g., content-defined chunking).
• Indexing: how to efficiently map file offsets → blocks and detect dirty/changed blocks.
• Performance trade‑offs:
  • CPU cost for computing hashes/checksums.
  • Memory overhead for block metadata.
• Failure handling:
  • How to retry or request missing/corrupted blocks.
  • How to reconcile when block metadata and file contents diverge.