`PdfSmartCopy` can take effectively unbounded time on nested shared Form XObjects
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- java
- Domain
- backend, performance
Research direction
Start in PdfSmartCopy.ByteStore, which the issue identifies as recursively traversing shared indirect objects during deduplication. Use the provided createPdfWithNestedFormXObjects example to reproduce the slowdown, then compare processing with PdfCopy. Done when PdfSmartCopy handles the shared nested Form XObjects without the reported excessive time and memory use.
Written by the indexing model from the issue text.
Description
Describe the bug
Using PdfSmartCopy to process a valid PDF with Form XObjects that reference nested resource dictionaries that repeatedly contain many Form XObjects from the next level can take a long time and use a lot of memory even for relatively small files. The problem seems to be that, in order to perform deduplication, PdfSmartCopy.ByteStore recursively traverses shared indirect objects once for every path through the resource graph, without keeping track of already-serialized indirect objects.
Important to note that using PdfCopy to process the same kind of file works fine, without taking too long or using too much memory.
To Reproduce
The document that triggered the issue cannot be shared because it contains private information. It is a 11 page document of roughly 300 KB, generated by the library TCPDF 6.7.5. The following code generates a 1 page pdf document that demonstrates the issue:
private void test() {
var pdfBytes = createPdfWithNestedFormXObjects(8, 8);
try (var doc = new Document(); var out = new ByteArrayOutputStream(); var copy = new PdfSmartCopy(doc, out)) {
doc.open();
copy(pdfBytes, copy);
}
}
void copy(byte[] bytes, PdfCopy copy) throws Exception {
try (var reader = new PdfReader(bytes)) {
for (var page = 1; page <= reader.getNumberOfPages(); page++) {
copy.addPage(copy.getImportedPage(reader, page));
}
}
}
private byte[] createPdfWithNestedFormXObjects(int nestedLeves, int fanout) throws Exception {
var output = new ByteArrayOutputStream();
var document = new Document(PageSize.A4);
var writer = PdfWriter.getInstance(document, output);
document.open();
document.add(new Paragraph("Test PDF, nested Form XObjects"));
List<PdfTemplate> layer = new ArrayList<>();
for (var index = 0; index < fanout; index++) {
var leaf = PdfTemplate.createTemplate(writer, 20, 20);
leaf.rectangle(0, 0, 10, 10);
leaf.stroke();
layer.add(leaf);
}
for (var level = 1; level < nestedLeves; level++) {
var parents = new ArrayList<PdfTemplate>();
for (var index = 0; index < fanout; index++) {
var parent = PdfTemplate.createTemplate(writer, 20, 20);
for (var nested : layer) {
parent.addTemplate(nested, 0, 0);
}
parents.add(parent);
}
layer = parents;
}
for (var form : layer) {
writer.getDirectContent().addTemplate(form, 0, 0);
}
document.close();
return output.toByteArray();
}
Varying the parameters of createPdfWithNestedFormXObjects allows to see the cases that trigger the problem.
Expected behavior
Using PdfSmartCopy should take more or less the same amount of time and use more or less the same amount of memory of using PdfCopy to process a document with multple levels of Form XObjects that reference shared Form XObjects.
System
- OpenPDF version: 3.0.3, 3.0.4, 3.0.5
- Dominant language
- Java
- Stars
- 4.4k
- Forks
- 714
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from LibrePDF/OpenPDF
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
IndexOutOfBoundsException in PdfReaderPossibly taken @andreasrosdalw claimed this 93 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Difficulty 4/5 1-2 days Newbie friendliness 45/100
All issues in LibrePDF/OpenPDF
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
oracle/graalvm-reachability-metadata#11318 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/maven-remote-resources-plugin#335 ·
Maintainers usually reply within 1 day
-
infrastructure P2 security
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
logisim-evolution/logisim-evolution#3138 ·
Maintainers usually reply within 1 day