Integrate the full PDF corpus into CI

Open
#280 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
18/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
java
Domain
ci-cd, testing

Research direction

This is a design-level task with no named files or tests yet. Start by reading the existing CI pipeline and the current ad hoc comparison scripts to see what runs today. Agree on the corpus access method, the 5% pixel-diff threshold and the baseline format with the maintainer before writing code. Done means a CI job runs the corpus, reports per-page diffs, and keeps failure artifacts.

Written by the indexing model from the issue text.

Description

I currently use around 70,000 PDFs for testing JPDFium, hosted on a storage VPS. Testing against this collection still relies on ad hoc scripts and manual "does this look right?" checks.

The goal is to integrate the VPS-hosted corpus into the existing CI pipeline and make these checks automated and reproducible. This should cover document loading, rendering, and relevant editing operations, with clear reports and retained failure artifacts to make regressions easier to investigate.

Notes on visual testing:

Compare rendered output against baseline images and flag differences above a threshold. For example, flag a page when more than 5% of its pixels differ, and include the baseline, actual output, and a diff image in the report.

Comparisons should account for the expected effects of each operation. For example, a redaction test should verify that the intended area changes while the rendered content outside it remains unchanged.

Notes on structural testing:

Run qpdf --check and similar validation tools on both the input and output PDFs, capturing their exit codes, warnings, and errors. Compare the results to see pre-existing problems in the source documents from new issues introduced by JPDFium.

These checks should complement visual and operation-specific tests. Passing qpdf --check (or the equivalent of other CLI PDF binaries) does not guarantee that a PDF renders correctly or that an editing operation produced the intended result (see: qpdf read the docs) BUT may give some hints when something is going in the wrong direction.

Dominant language
Java
Stars
16
Forks
4
Avg merge
2d 2h
Merged PRs (30d)
56

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Stirling-Tools/JPDFium

All issues in Stirling-Tools/JPDFium

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.