Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Silent data-correctness bug: `readinto()` / `readinto1()` on `stream_reader` return `tell() == 0` after successful reads

Aperta Adatta ai principianti
  • Qualcosa non va?
#295 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
76/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Tranquilla
Stack tecnologico
c, python
Ambito
backend

Direzione di ricerca

Inizia in c-ext/compressionreader.c, nei corpi dei metodi readinto e readinto1, confrontando il loro ZSTD_outBuffer locale con la gestione persistente dell'output del percorso read(). Esegui il riproduttore Python fornito e verifica che tell() dopo readinto() e readinto1() avanzi in modo coerente con read(); esamina il pattern sul lato della decompressione se pertinente.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

ZstdCompressionReader.readinto(buf) and .readinto1(buf) correctly copy compressed bytes into the caller's buffer, but the reader's internal bytesCompressed counter is not updated. As a result, stream_reader.tell() returns 0 after any number of successful readinto() calls, even though bytes were in fact written. The equivalent .read() path correctly advances the counter.

Impact

  • Severity: Silent data-correctness bug — no crash, no exception, just a wrong value from tell(). Any caller that relies on tell() to measure progress, compute offsets, or compare positions will silently malfunction.
  • Reachability: Standard io.RawIOBase idioms — any use of readinto() / readinto1() on a stream_reader. Common in performance-sensitive decode pipelines that reuse a pre-allocated buffer.
  • Version: 0.25.0 (commit 7a77a75).
  • Platform: Platform-independent.

Reproducer

import zstandard, io

data = b'hello world ' * 10000

# read() — tell() works correctly
comp1 = zstandard.ZstdCompressor()
r1 = comp1.stream_reader(io.BytesIO(data))
r1.__enter__()
while r1.read(1024):
    pass
print("read     tell:", r1.tell())       # 29 (correct: total compressed bytes)
r1.__exit__(None, None, None)

# readinto() — tell() stuck at 0
comp2 = zstandard.ZstdCompressor()
r2 = comp2.stream_reader(io.BytesIO(data))
r2.__enter__()
buf = bytearray(1024)
while r2.readinto(buf):
    pass
print("readinto tell:", r2.tell())       # 0 — BUG (should match read() path)
r2.__exit__(None, None, None)

Root cause

readinto / readinto1 build a ZSTD_outBuffer on the stack that wraps the caller's buffer:

ZSTD_outBuffer output = {dest, dest_size, 0};
zresult = ZSTD_compressStream2(cctx, &output, &input, ZSTD_e_continue);

After the call, output.pos holds the number of bytes written to dest. The reader then updates its position by reading from self->output.pos — the persistent struct, which the local-struct call never touched. So self->output.pos stays at zero, and bytesCompressed never advances.

The read() path uses self->output directly (not a local copy), so the persistent field is updated by ZSTD_compressStream2. That's why tell() works after read() but not after readinto.

Affected sites

  • c-ext/compressionreader.c — readinto and readinto1 method bodies.

(Same pattern may warrant a look on the decompression side as well, although the main analysis only flagged compression.)

Suggested fix

Two options; either is minimal.

Option A — advance from the local struct

Read bytesCompressed from the local output.pos before it goes out of scope:

ZSTD_outBuffer output = {dest, dest_size, 0};
zresult = ZSTD_compressStream2(cctx, &output, &input, ZSTD_e_continue);
/* ... */
self->bytesCompressed += output.pos;   /* was: effectively + self->output.pos, i.e. 0 */
Option B — share the persistent struct

If you'd rather the two methods share the read() bookkeeping path, use self->output directly instead of a stack-local struct, and update self->output.dst / self->output.size to point at the caller's buffer before the call. Slightly more invasive but avoids duplicating the position-update logic.

Methodology

Found via cext-review-toolkit (Tree-sitter-based static analysis with structured naive/informed review passes). Reproducer verified live on CPython 3.14.3 debug build — read() path returns tell() == 29 (matches the compressed-output length); readinto() path returns tell() == 0 after the exact same compressed sequence is consumed. Happy to open a PR.

Discovery, root-cause analysis, and issue drafting were performed by Claude Code and reviewed by a human before filing.

Full report

Complete multi-agent analysis (48 FIX findings across 13 categories, plus a reproducer appendix): https://gist.github.com/devdanzin/b86039ac097141579590c1a0f3a43605

Lingua principale
C
Stelle
641
Fork
117
Merge medio
1g 14h
PR unite (30g)
5

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di indygreg/python-zstandard

Tutte le issue di indygreg/python-zstandard

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.