Proposal: attach open-access full-text PDFs from OpenAlex

Comienza leyendo app/models/jobs.py y los puntos de entrada existentes para revisar, aplicar y revertir trabajos; después, compara la implementación hermana de doi_resolver. Empieza por el trabajo dry-run propuesto: estará terminado cuando informe de cada DOI, la URL candidata, la versión detectada, la licencia y la confianza, sin subir archivos.

Idea

OpenAlex already tells us, for most publications, where a free and legal full text lives — it carries Unpaywall's open-access data. BackToPure downloads that data today and ignores it: it writes the citation metadata into Pure and drops the PDF link.

Proposal: also fetch the PDF and attach it to the Pure record.

Depends on #2, #3 and #5 — build this as a job type in the new architecture, not on the legacy path.

We already have a working implementation to draw on

A sibling project, doi_resolver, already does this against Pure. It is proven code and should be the starting point rather than a fresh build.

• Upload endpoint: PUT {base}/research-outputs/file-uploads, Content-Type: application/pdf, raw bytes as the body (not multipart). Returns JSON containing a key.
• That key is then referenced back on the record as a FileElectronicVersion entry in electronicVersions.
• It also has the safety work done: HTML-landing-page preflight, %PDF- magic-byte and content-type validation, a size cap, a direct/likely/weak scorer for candidate links, and mappers to Pure's access-type and licence URIs.

Roughly 150 of the relevant ~250 lines are dependency-free pure functions that port almost verbatim. The remainder is httpx-async I/O to rewrite as synchronous requests.

Scope: existing records as well as new ones

An earlier draft of this issue framed BackToPure as only creating records that do not yet exist in Pure, and doi_resolver as only enriching ones that do. That split is not a reason to limit this work.

Attaching PDFs is worth doing in both directions, and they are close to the same code:

1. New records — attach during the research-outputs import, after the record is created.
2. Existing records — a separate job that walks publications already in Pure, finds those without a full text, and attaches one.

(2) is arguably the larger win: the back-catalogue of existing Pure records is far bigger than whatever a given import adds. It also needs no record creation at all, so it is the simpler of the two to build and to review.

Suggest these become two job types sharing one module, or one job type with a scope parameter. Decide during implementation.

Why port rather than depend on doi_resolver

doi_resolver is a FastAPI service. Depending on it would add fastapi, uvicorn, httpx, pydantic and pydantic-settings to BackToPure's four-line requirements.txt, plus a second .env config layer alongside config.ini. That is a real install and configuration cost for people who only want BackToPure. The two projects also do different halves of the job: doi_resolver attaches to records that already exist in Pure, BackToPure creates records that do not.

Accepted cost: the scorer and mappers will exist in both repositories. They are small and dependency-free; extracting a shared package can wait until it actually hurts.

The correctness issue to fix while porting

doi_resolver hardcodes versionType to publishersversion / "Final published version", because its candidate model has no version field.

This is wrong often enough to matter. Unpaywall's best OA location is frequently a repository copy — an accepted manuscript or a preprint. Labelling an accepted manuscript as the final published version is a metadata error that repository managers will rightly object to.

BackToPure can do better here with data it already downloads: OpenAlex's best_oa_location.version gives publishedVersion / acceptedVersion / submittedVersion, alongside pdf_url, license and is_oa. Map that to Pure's versionType instead of hardcoding it.

Fit with the job architecture

This maps onto the existing model cleanly:

• A new JobType in app/models/jobs.py with its own JobTypeDefinition (script_path, artifact_dir, identity_columns=("doi",)).
• The NEEDS_REVIEW status and /api/jobs/<id>/review-table give the human check on version and licence before anything is deposited. This should not be fully automatic on day one.
• /api/jobs/<id>/apply and /rollback give the apply hook and, importantly, a rollback path — which matters when the operation deposits files rather than metadata.
• Job logs, artifacts and cancellation are already handled, which covers the resumability problem for what will be a long-running, network-flaky job.

Open questions — policy, not code

These need an answer from the Pure/repository side before implementation:

1. Which versions may be auto-deposited? Publisher version only, or also accepted manuscripts?
2. Who owns the licence check? Depositing a file is a different act from recording a citation.
3. What about duplicates with the existing OA workflow — if UU already holds the file, should this skip?
4. Storage growth in Pure at faculty scale.

Suggested first step

A dry-run job type that reports what would be attached — DOI, candidate URL, detected version, licence, confidence — without uploading anything. That gives the repository team something concrete to set policy against, and is useful on its own as a coverage report.