[Bug]: Crawl4aiDockerClient cannot send 0.9 declarative hooks, and returns result.pdf as base64 text

Abierto
#2,483 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
70/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
docker, python
Área
api

Línea de trabajo

Start in the Crawl4aiDockerClient class, in its crawl() method, where hooks are currently wrapped as {"hooks": {"code": ..., "timeout": N}} and where CrawlResult(**r) is built on both /crawl and /crawl/stream. Check GET /hooks/info and deploy/docker/api.py for the 0.9 declarative format and the base64 pdf encoding. Done means: a list of {action, params} is sent as hooks.hooks, dict hooks still use the legacy path, the server's hooks warning is raised as a UserWarning, and result.pdf is decoded to raw PDF bytes on both paths.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

⚙ Done 🐞 Bug
crawl4ai version

0.9.5 (also on develop)

Expected Behavior
  1. Crawl4aiDockerClient.crawl(..., hooks=...) can use the hooks the 0.9 server supports: declarative actions such as {"action": "block_resources", "params": {...}} (listed by GET /hooks/info).
  2. With CrawlerRunConfig(pdf=True), result.pdf holds the raw PDF bytes (b'%PDF-1.4 ...'), the same as AsyncWebCrawler.
Current Behavior

Hooks. The client only knows the pre-0.9 inline-code hooks. It always sends {"hooks": {"code": {...}, "timeout": N}}, which the 0.9 server never runs:

  • Hooks disabled (the default): RequestError: Server error 403: Inline hook code (hooks.code) was removed in 0.9.0 and cannot be enabled; it was not executed. ...
  • Hooks enabled (CRAWL4AI_HOOKS_ENABLED=true): the crawl returns success=True, the hook does not run, and the server's warning (results["hooks"]["warning"], or the X-Hooks-Warning header on /crawl/stream) is dropped by the client. The user is never told.

Passing the declarative format crashes in the client before any request:

AttributeError: 'list' object has no attribute 'values'

The crawl() docstring still shows Python function hooks as the example.

PDF. The server base64-encodes pdf for JSON (deploy/docker/api.py, b64encode(result_dict['pdf']), since 0.7.2), and the client passes that string straight into CrawlResult(**r). Pydantic turns the string into bytes without decoding, so result.pdf is base64 text (b'JVBERi0xLjQK...'). A file written from it is not a valid PDF, and no error is raised. Both /crawl and /crawl/stream are affected.

Is this reproducible?

Yes

Code snippets
from crawl4ai import CrawlerRunConfig
from crawl4ai.docker_client import Crawl4aiDockerClient

async with Crawl4aiDockerClient(base_url="http://localhost:11235") as client:
    client._http_client.headers["Authorization"] = f"Bearer {token}"

    # 0.9 declarative hooks: crashes in the client
    await client.crawl(
        ["https://httpbin.org/headers"],
        hooks=[{"action": "set_headers", "params": {"headers": {"X-Test": "1"}}}],
    )
    # AttributeError: 'list' object has no attribute 'values'

    # Legacy function hooks on a server with hooks enabled: success, hook never runs, no warning
    async def before_goto(page, **kwargs):
        await page.set_extra_http_headers({"X-Test": "1"})
        return page

    result = await client.crawl(["https://httpbin.org/headers"], hooks={"before_goto": before_goto})
    print(result.success, "X-Test" in result.html)  # True False

    # PDF comes back as base64 text
    result = await client.crawl(["https://example.com"], crawler_config=CrawlerRunConfig(pdf=True))
    print(result.pdf[:10])  # b'JVBERi0xLj'  (local AsyncWebCrawler: b'%PDF-1.4\n%')
Proposed fix

In Crawl4aiDockerClient:

  • send a list of {"action", "params"} as {"hooks": {"hooks": [...], "timeout": hooks_timeout}};
  • keep sending a dict the legacy way, so servers older than 0.9 still run it;
  • raise the server's hooks warning as a UserWarning on both /crawl and /crawl/stream;
  • decode pdf with strict base64 before building CrawlResult, on both paths, leaving any non-base64 value unchanged (screenshot stays as it is: it is base64 text in local crawls too).

I'll open a PR.

OS

macOS 26.6 (platform-independent)

Python version

3.13

Lenguaje dominante
Python
Estrellas
87.2k
Forks
9.1k
Merge medio
1 d 6 h
PR fusionados (30 d)
25

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de unclecode/crawl4ai

Todos los issues de unclecode/crawl4ai

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.