Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Bug]: PDF URI links are silently discarded by PDFContentScrapingStrategy due to Links schema mismatch

Aperta
  • Qualcosa non va?
#2,374 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@Hugo-DDT ci sta già lavorando.

Dal 15/10/2026.

  • #2375 di @Hugo-DDT — aperta

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
data

Direzione di ricerca

Start in crawl4ai/processors/pdf/__init__.py at PDFContentScrapingStrategy.scrap(), where the {"urls": []} dictionary is built, then compare it with Links, Link and ScrapingResult in crawl4ai/models.py. Reproduce with the snippet in the issue, which should show URI links present in processor.py output but missing from scraped.links. The maintainer has not yet confirmed the link representation, so agree on the contract before changing it. Done means a generated PDF's URI link appears in ScrapingResult.links with its page preserved.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

crawl4ai version

0.9.5 (also observed in 0.9.4).

Expected Behavior

URI link annotations extracted from a PDF should survive PDFContentScrapingStrategy.scrap() and be accessible in the returned ScrapingResult.links, with their source page numbers preserved or represented by an explicitly documented contract.

This report concerns preservation of the extracted link inventory, not reconstructing clickable text positions in HTML/Markdown or PDF-internal page destinations.

Current Behavior

The public processor extracts the URI correctly, but the public PDF scraping strategy returns success=True and empty internal / external link lists for the same file.

The apparent cause is a schema mismatch:

  • PDFContentScrapingStrategy.scrap() builds links = {"urls": []} and appends {"url": link, "page": page.page_number}.
  • It passes that dictionary to ScrapingResult.
  • ScrapingResult.links is typed as Links, which defines only internal and external. Its Link items use href rather than url and have no explicit PDF page field.
  • The unrecognized urls key is silently ignored during model construction, losing the collected links.

Sources: PDF scraper, models, and PDF processor.

The PDF parsing documentation also describes links["urls"], suggesting that preserving PDF links is intended, but the current result model cannot represent that structure.

Is this reproducible?

Yes.

Inputs Causing the Bug

A local, generated one-page PDF containing one URI annotation. No browser, network requests, external files, credentials, or application-specific adapter is needed. The page is intentionally blank apart from its annotation to isolate link preservation.

Steps to Reproduce
  1. Use Python 3.12 and install pip install "crawl4ai[pdf]==0.9.5".
  2. Save the following snippet as repro_pdf_links.py.
  3. Run python repro_pdf_links.py.
  4. Observe that the processor contains the URI but the scraping result has empty link lists.
Code snippets
from pathlib import Path
from tempfile import TemporaryDirectory

from pypdf import PdfWriter
from pypdf.annotations import Link

from crawl4ai.processors.pdf import PDFContentScrapingStrategy
from crawl4ai.processors.pdf.processor import NaivePDFProcessorStrategy

with TemporaryDirectory() as folder:
    path = Path(folder) / "linked.pdf"
    writer = PdfWriter()
    writer.add_blank_page(width=612, height=792)
    writer.add_annotation(
        0,
        Link(rect=(20, 20, 200, 40), url="https://example.org/reference"),
    )
    writer.write(path)

    processed = NaivePDFProcessorStrategy(extract_images=False).process(path)
    scraped = PDFContentScrapingStrategy(extract_images=False).scrap(str(path), "")

    print("processor links:", processed.pages[0].links)
    print("scraping success:", scraped.success)
    print("scraping links:", scraped.links.model_dump())
OS

macOS 27.0.1, Apple Silicon (arm64).

Python version

3.12.13; pypdf 6.20.1.

Browser

Not applicable; the reproduction uses a local PDF and public SDK methods directly.

Browser version

Not applicable.

Error logs & Screenshots (if applicable)

No exception is raised. Actual output on 0.9.5:

processor links: ['https://example.org/reference']
scraping success: True
scraping links: {'internal': [], 'external': []}
Proposed direction / contribution

I would like to contribute a focused fix targeting develop, with regression coverage using a generated PDF and updated documentation. Before changing the public link contract, could you confirm the intended representation?

One possible approach is to map PDF URI links into the existing internal / external structure using href, and add an optional 1-based page field to Link so PDF page attribution is not lost. For remote PDFs, classification could reuse the existing URL classification rules; for local PDFs without a source URL, treating URI links as external is a possible fallback, subject to maintainer preference.

This is a proposal rather than an assumed API decision. Reader reuse / parallel parsing optimization and reconstruction of inline clickable text would remain outside this fix.

Lingua principale
Python
Stelle
85.4k
Fork
8.9k
Merge medio
6g 5h
PR unite (30g)
24

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di unclecode/crawl4ai

Tutte le issue di unclecode/crawl4ai

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.