ipfs/kubo
在 GitHub 查看enhancement: tarsnap chunker / importer (Content defined, polynomial, fast converging chunking)
Open
#3,603 建立於 2017年1月17日
help wantedkind/enhancement
倉庫指標
- Star
- (13,906 star)
- PR 合併指標
- (平均合併 4天 2小時) (30 天內合併 28 個 PR)
描述
Version information:
go-ipfs version: 0.4.4
Type: Feature, Enhancement
Priority: P4
Area: Tools, Importer
Description:
Better suited for maximizing deduplication ratio then current Rabin chunker.
Using smaller chunks with faster convergence yields greater space savings, and the benefit depending on dataset can be great in comparison to Rabin.
The mean chunk size used by tarsnap is 64k.
Source: https://github.com/Tarsnap/tarsnap/blob/master/tar/multitape/chunkify.h https://github.com/Tarsnap/tarsnap/blob/master/tar/multitape/chunkify.c
Related: https://moinakg.wordpress.com/2012/11/11/inside-content-defined-chunking-in-pcompress/ https://moinakg.wordpress.com/2012/11/15/inside-content-defined-chunking-in-pcompress-part-2/