Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

[Bug]: PDF URI links are silently discarded by PDFContentScrapingStrategy due to Links schema mismatch

未关闭
  • 哪里不对？
#2,374 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 2 天内回复

还没有人认领这个 Issue。

评估

难度
3/5
预计耗时
1-2 天
新手友好度
35/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
python
领域
data

调研方向

Start in crawl4ai/processors/pdf/__init__.py at PDFContentScrapingStrategy.scrap(), where the {"urls": []} dictionary is built, then compare it with Links, Link and ScrapingResult in crawl4ai/models.py. Reproduce with the snippet in the issue, which should show URI links present in processor.py output but missing from scraped.links. The maintainer has not yet confirmed the link representation, so agree on the contract before changing it. Done means a generated PDF's URI link appears in ScrapingResult.links with its page preserved.

由索引模型根据 Issue 内容生成。

描述

⚙ Done 🐞 Bug
crawl4ai version

0.9.5 (also observed in 0.9.4).

Expected Behavior

URI link annotations extracted from a PDF should survive PDFContentScrapingStrategy.scrap() and be accessible in the returned ScrapingResult.links, with their source page numbers preserved or represented by an explicitly documented contract.

This report concerns preservation of the extracted link inventory, not reconstructing clickable text positions in HTML/Markdown or PDF-internal page destinations.

Current Behavior

The public processor extracts the URI correctly, but the public PDF scraping strategy returns success=True and empty internal / external link lists for the same file.

The apparent cause is a schema mismatch:

  • PDFContentScrapingStrategy.scrap() builds links = {"urls": []} and appends {"url": link, "page": page.page_number}.
  • It passes that dictionary to ScrapingResult.
  • ScrapingResult.links is typed as Links, which defines only internal and external. Its Link items use href rather than url and have no explicit PDF page field.
  • The unrecognized urls key is silently ignored during model construction, losing the collected links.

Sources: PDF scraper, models, and PDF processor.

The PDF parsing documentation also describes links["urls"], suggesting that preserving PDF links is intended, but the current result model cannot represent that structure.

Is this reproducible?

Yes.

Inputs Causing the Bug

A local, generated one-page PDF containing one URI annotation. No browser, network requests, external files, credentials, or application-specific adapter is needed. The page is intentionally blank apart from its annotation to isolate link preservation.

Steps to Reproduce
  1. Use Python 3.12 and install pip install "crawl4ai[pdf]==0.9.5".
  2. Save the following snippet as repro_pdf_links.py.
  3. Run python repro_pdf_links.py.
  4. Observe that the processor contains the URI but the scraping result has empty link lists.
Code snippets
from pathlib import Path
from tempfile import TemporaryDirectory

from pypdf import PdfWriter
from pypdf.annotations import Link

from crawl4ai.processors.pdf import PDFContentScrapingStrategy
from crawl4ai.processors.pdf.processor import NaivePDFProcessorStrategy

with TemporaryDirectory() as folder:
    path = Path(folder) / "linked.pdf"
    writer = PdfWriter()
    writer.add_blank_page(width=612, height=792)
    writer.add_annotation(
        0,
        Link(rect=(20, 20, 200, 40), url="https://example.org/reference"),
    )
    writer.write(path)

    processed = NaivePDFProcessorStrategy(extract_images=False).process(path)
    scraped = PDFContentScrapingStrategy(extract_images=False).scrap(str(path), "")

    print("processor links:", processed.pages[0].links)
    print("scraping success:", scraped.success)
    print("scraping links:", scraped.links.model_dump())
OS

macOS 27.0.1, Apple Silicon (arm64).

Python version

3.12.13; pypdf 6.20.1.

Browser

Not applicable; the reproduction uses a local PDF and public SDK methods directly.

Browser version

Not applicable.

Error logs & Screenshots (if applicable)

No exception is raised. Actual output on 0.9.5:

processor links: ['https://example.org/reference']
scraping success: True
scraping links: {'internal': [], 'external': []}
Proposed direction / contribution

I would like to contribute a focused fix targeting develop, with regression coverage using a generated PDF and updated documentation. Before changing the public link contract, could you confirm the intended representation?

One possible approach is to map PDF URI links into the existing internal / external structure using href, and add an optional 1-based page field to Link so PDF page attribution is not lost. For remote PDFs, classification could reuse the existing URL classification rules; for local PDFs without a source URL, treating URI links as external is a possible fallback, subject to maintainer preference.

This is a proposal rather than an assumed API decision. Reader reuse / parallel parsing optimization and reconstruction of inline clickable text would remain outside this fix.

主要语言
Python
星标
85.9k
派生
8.9k
平均合并
4 天 20 分钟
30 天内合并 PR
19

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

unclecode/crawl4ai 的其他 Issue

查看 unclecode/crawl4ai 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。