Reading a PDF can use unbounded memory: the decoder and the renderer have no limit
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 35/100
Direzione di ricerca
Start with the PDF reader in pdf.rs, then inspect how hayro 0.9 handles FlateDecode streams, nested Form XObjects, and image decoding; the issue names memory:: limits and progress::check as possible integration points. The issue asks for a direction among three approaches, so first compare their coverage and platform limits. Done means PDF decoding and drawing have enforceable resource limits and can stop safely under the relevant memory and cancellation constraints.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
The native PDF reader (#2352) caps the file at 1 GB, but nothing caps what decoding and drawing it takes. A small hostile PDF can make RustGIS ask for gigabytes, and when that allocation fails the process aborts. catch_unwind can't catch an allocation failure.
What I measured on main, with a test allocator capping the heap (pdf.rs and hayro 0.9.0 are unchanged as of e186970d, 2026-11-19):
- A FlateDecode content or object stream of zeros: a 2 MB file peaks at 4.1 GB. Under a 300 MB cap, a 1 MB file aborts ("memory allocation of 536870912 bytes failed"). hayro-syntax inflates with
read_to_endinto an unboundedVec. - Form XObjects that each paint the next one twice: every two levels of nesting cost 4× the time and memory (depth 20: 13 s and 450 MB from a 4 KB file). A 9 KB file at depth 50 aborts. hayro caps nesting depth at 50 but not total work, and
hayro::renderhas no cancel hook.
Images are the third case. A 631-byte JPEG whose dictionary declares 65535×65535 makes the decoder ask for 12.9 GB. I tried refusing oversized images in RustGIS before drawing, by walking the page's images and reading each codec's header. I dropped it, because checking from outside hayro kept disagreeing with how hayro reads the same data:
- it refused maps that open fine today (hayro already shrinks a JPEG header that claims more than its dictionary);
- it refused pages over images they never draw, or over one damaged thumbnail;
- it decoded every content stream twice, with no Cancel;
- it could still be bypassed, for example by JBIG2 page-information order.
So all three cases come down to the same thing: hayro 0.9 has no setting to limit decoded size or drawing work.
Separately, I have fixes for what the reader drops without saying on my fork (branch fix/pdf-reader-says-what-it-drops), and I'll open them as their own PR. They don't bound memory. They:
- say how often an image couldn't be decoded, or text was drawn in a stand-in font;
- name fonts that aren't in the file;
- say when a page's coordinates are in the OGC
/LGIDictform, which RustGIS doesn't read; - read UTF-16 and PDFDocEncoding coordinate-system text (ISO 32000-1, Table D.2).
Options I can see:
- Add a decoded-size budget and an operation or cancel hook to hayro upstream, then set them from
memory::limits andprogress::check. - Drive
hayro_interpret::interpret_pagethrough our ownDevicethat counts drawing work and checks Cancel. That covers the forms case but not inflation. - Draw in a child process with a memory limit, which can be killed on Cancel. That covers both, but desktop only.
Which way would you like to go? I'm happy to do the work once there's a direction.
- Lingua principale
- Rust
- Stelle
- 1.9k
- Fork
- 268
- Merge medio
- 11h 1m
- PR unite (30g)
- 942
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di opengeos/RustGIS
-
[Bug]: layer.move to the place a layer already has fails and stops a script; with several layers it leaves an empty undo stepForse già presa @jgregoriorincon l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
opengeos/RustGIS#2621 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 85/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 61/100
opengeos/RustGIS#1446 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 52/100
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di opengeos/RustGIS
Issue simili
-
文件夹重大BUG:不支持重命名Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
I maintainer di solito rispondono entro 1 giorno
-
postgresql-rules / queue-rules: implicit KEY SHARE before a parent key update, and job self-overlapAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
jonathanong/no-mistakes#3879 ·
I maintainer di solito rispondono entro 1 giorno
-
area:index v0.35.0
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
I maintainer di solito rispondono entro 1 giorno
-
needs triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
pbakaus/impeccable#1653 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 80/100
nautechsystems/nautilus_trader#5741 ·
I maintainer di solito rispondono entro 1 giorno