ipfs/kubo

merkledag: CumulativeSize on directories overflows

Open

#3440 opened on Nov 29, 2016

View on GitHub
 (4 comments) (0 reactions) (0 assignees)Go (13,906 stars) (2,725 forks)batch import
help wantedkind/bugstatus/deferred

Description

Version information: 0.4.5

Type: bug

Priority: P4

Description:

CumulativeSize of a directory (or file) can reach ~16 EB maximum, after which it wraps around to 0B. An example showing this overflow: QmW4Zd911qv7uaRM2vi79YsfHvZ93skxiCxLnwpgpf7hp3 The directory looks like 11GB, but contains a lot of 'large' folders (like a zip bomb) + a data folder (here, the npm repo) of 723 GB. Here's the folder bomb used.

Contributor guide

Tech stack
go
Domain
backend
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
understanding of Go integer overflowfamiliarity with IPFS merkledag
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
The bug is a uint64 overflow in CumulativeSize when directory sizes exceed ~16 EB. To reproduce, use the provided IPFS hashes. The fix likely involves checking for overflow in the size accumulation logic in the merkledag package or switching to a larger integer type. Look at the implementation of CumulativeSize in the go merkledag or go unixfs packages.