Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Bug]: Same PDF gives different markdown on each run when batch_size > 1 (threads share current_page_number)

Cerrado Apto para principiantes
  • ¿Algo no está bien?
#2,319 4 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 3 días

@Kylinny ya está trabajando en esto.

Desde el 30/9/2026.

  • #2320 de @Kylinny — abierto

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
84/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
backend

Línea de trabajo

Empieza en processors/pdf/processor.py, en NaivePDFProcessorStrategy.process_batch(), _process_page() y _extract_images(); revisa cómo se utiliza la current_page_number compartida. Ejecuta repro.py con batch_size=4 y batch_size=1, y verifica que las ejecuciones repetidas sean deterministas, que el formato de la página 1 se mantenga en la página 1 y que los números de página y los nombres de las imágenes sigan siendo correctos.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

⚙ Done 🐞 Bug 🥸 - Medium
crawl4ai version

0.9.0 (also reproduced with processors/pdf/processor.py and utils.py from v0.9.4; the code involved is unchanged on main)

Expected Behavior

This report was drafted with the help of an AI assistant. The reproduction below was run locally.

Processing the same PDF twice should give the same markdown, and the page-1-only formatting in clean_pdf_text() should only ever apply to page 1.

Current Behavior

With the default batch_size=4 (also the default of PDFContentScrapingStrategy), the output changes from run to run for the same file. In the repro below, a Title Case line that clean_pdf_text() treats as an author line (which it only does on page 1) ends up bolded on page 1, on no page at all, or on other pages, giving 5 to 7 distinct outputs over 30 runs. With batch_size=1 the output is always the same. We noticed because we hash the extracted text to detect changed documents, and unchanged PDFs looked changed on every crawl.

The cause is in NaivePDFProcessorStrategy.process_batch(). Each worker writes its page number to the shared processor instance, and _process_page() reads it back after text extraction, by which time another worker may have overwritten it:

def process_page_safely(page_num: int):
    ...
    self.current_page_number = page_num + 1
    return self._process_page(page, image_dir)

# _process_page()
pdf_page.markdown = clean_pdf_text(self.current_page_number, pdf_page.raw_text)
pdf_page.html = clean_pdf_text_to_html(self.current_page_number, pdf_page.raw_text)

The same attribute also feeds PDFPage.page_number and the extracted image file names (page_{current_page_number}_img_{n}).

Suggested fix: pass the page number down instead of storing it on self, i.e. self._process_page(page, image_dir, page_num + 1), and use that argument in _process_page() and _extract_images(). With that change the repro gives a single output for batch_size=4 as well.

Workaround: PDFContentScrapingStrategy(batch_size=1).

Is this reproducible?

Yes

Inputs Causing the Bug
- URL(s): none, the script below generates a local 12-page PDF
- Settings used: NaivePDFProcessorStrategy(batch_size=4), the default, vs batch_size=1
- Input data: each page has a Title Case line that clean_pdf_text() only bolds on page 1
Steps to Reproduce
1. pip install "crawl4ai[pdf]" fpdf2
2. Save the code snippet below as repro.py
3. python repro.py
4. Compare the batch_size=4 and batch_size=1 results
Code snippets
# pip install "crawl4ai[pdf]" fpdf2
from collections import Counter
from pathlib import Path

from fpdf import FPDF

from crawl4ai.processors.pdf.processor import NaivePDFProcessorStrategy

# 12 pages, each with a Title Case line that clean_pdf_text() only bolds on page 1.
path = Path("race.pdf")
pdf = FPDF()
pdf.set_font("Helvetica", size=12)
for n in range(1, 13):
    pdf.add_page()
    pdf.cell(0, 10, f"Chapter Heading Number {n}", new_x="LMARGIN", new_y="NEXT")
    pdf.cell(0, 10, "Assessment Overview", new_x="LMARGIN", new_y="NEXT")
    pdf.multi_cell(0, 8, f"this is the body of page {n}. " * 60)
pdf.output(str(path))

for batch_size in (4, 1):
    outputs = Counter()
    for _ in range(30):
        result = NaivePDFProcessorStrategy(batch_size=batch_size).process_batch(path)
        bolded = tuple(
            number
            for number, page in enumerate(result.pages, start=1)
            if "**Assessment Overview**" in page.markdown
        )
        outputs[bolded] += 1
    print(f"batch_size={batch_size}: {len(outputs)} distinct outputs over 30 runs")
    for bolded, runs in outputs.most_common():
        print(f"  {runs:2d} runs: bolded on pages {list(bolded)}")
OS

Linux (Arch Linux)

Python version

3.12.12

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

Output of repro.py on 0.9.0:

batch_size=4: 7 distinct outputs over 30 runs
  18 runs: bolded on pages [1]
   7 runs: bolded on pages []
   1 runs: bolded on pages [4]
   1 runs: bolded on pages [5]
   1 runs: bolded on pages [1, 3]
   1 runs: bolded on pages [1, 2]
   1 runs: bolded on pages [2]
batch_size=1: 1 distinct outputs over 30 runs
  30 runs: bolded on pages [1]
Lenguaje dominante
Python
Estrellas
85.9k
Forks
8.9k
Merge medio
5 d 6 h
PR fusionados (30 d)
24

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de unclecode/crawl4ai

Todos los issues de unclecode/crawl4ai

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.