Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

diff: two large files abort with a ~100 TB allocation

Aperta
#287 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
rust
Ambito
cli

Direzione di ricerca

The issue is in src/ed_diff.rs and related files (unified_diff.rs, normal_diff.rs, context_diff.rs, side_diff.rs) where diff::slice is called. First, understand the diff crate's algorithm and its O(n*m) allocation. Look for existing linear-space Myers implementations in Rust or consider patching the diff crate dependency. The goal is to replace the current call with an algorithm that handles large files without excessive memory, and ensure proper error handling instead of aborting. Test with the provided large file example to verify the fix.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

diff builds its line-level LCS with diff::slice() from the diff crate, which allocates an O(n·m) table over the two line vectors. For two
multi-million-line files that table is hundreds of terabytes, the allocation fails, and the process aborts with no diff:-prefixed diagnostic. GNU diff uses a linear-space Myers algorithm and handles the same files in the normal way.

$ python3 -c "open('a','w').write('\n'*5000000)"       # 5 MB, 5M empty lines
$ python3 -c "open('b','w').write('z\n'*5000000)"      # 10 MB, 5M 'z' lines
$ diff a b > /dev/null
memory allocation of 100000040000004 bytes failed
Aborted (core dumped)
$ echo $?
134

Every output mode fails the same way — the allocation happens before any
formatting:

$ for m in "" -u -c -e -y; do diff $m a b >/dev/null; echo "$m -> $?"; done
 -> 134
-u -> 134
-c -> 134
-e -> 134
-y -> 134

Root cause

Each output mode drives the same routine:

// src/ed_diff.rs:74   (identically: unified_diff.rs:68, normal_diff.rs:57,
//                      context_diff.rs:80, side_diff.rs:351)
for result in diff::slice(&expected_lines, &actual_lines) {

diff::slice is the diff crate's LCS over two slices; its dynamic-programming table is proportional to expected_lines.len() * actual_lines.len(). With 5M lines on each side that is ~2.5·10¹³ cells — the observed request is 100,000,040,000,004 bytes (~100 TB). Nothing bounds the input size before the call, and the allocation failure is an abort rather than an error the caller can report.

Lingua principale
Rust
Stelle
276
Fork
39
Merge medio
4g 12h
PR unite (30g)
3

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di uutils/diffutils

Tutte le issue di uutils/diffutils

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.