[Bug]: Wrong error message on failed crawling with CacheMode.ENABLED

Abierto
#2,490 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
Medio día
Aptitud para principiantes
56/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
playwright, python
Área
backend

Línea de trabajo

The traceback names _crawl_web and the MarkdownGenerationResult pydantic model, and the failed navigation is a Playwright Page.goto error. Start by reproducing with an unreachable URL under CacheMode.ENABLED and again under CacheMode.DISABLED, then compare the error fields. Done means the cached path reports the real navigation failure instead of the empty-dict validation error.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

⚙ Done 🐞 Bug
crawl4ai version

0.9.5

Expected Behavior

CrawlerRunConfig(cache_mode=CacheMode.ENABLED,...)

AsyncWebCrawler.arun(...)
"error": "All proxies failed: Failed on navigating ACS-GOTO:\nPage.goto: net::ERR_CONNECTION_REFUSED at https://some-wrong-site.com/\nCall log:\n - navigating to "https://some-wrong-site.com/\", waiting until "domcontentloaded"\n"

CrawlerRunConfig(cache_mode=CacheMode.DISABLED,...)

AsyncWebCrawler.arun(...)
"error": "All proxies failed: Failed on navigating ACS-GOTO:\nPage.goto: net::ERR_CONNECTION_REFUSED at https://some-wrong-site.com/\nCall log:\n - navigating to "https://some-wrong-site.com/\", waiting until "domcontentloaded"\n"

Current Behavior

CrawlerRunConfig(cache_mode=CacheMode.ENABLED,...)

AsyncWebCrawler.arun(...)
"error": "Unexpected error in _crawl_web at line 263 in init (../.venv/lib/python3.13/site-packages/pydantic/main.py):\nError: 3 validation errors for MarkdownGenerationResult\nraw_markdown\n Field required [type=missing, input_value={}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.13/v/missing\nmarkdown_with_citations\n Field required [type=missing, input_value={}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.13/v/missing\nreferences"

CrawlerRunConfig(cache_mode=CacheMode.DISABLED,...)

AsyncWebCrawler.arun(...)
"error": "All proxies failed: Failed on navigating ACS-GOTO:\nPage.goto: net::ERR_CONNECTION_REFUSED at https://some-wrong-site.com/\nCall log:\n - navigating to "https://some-wrong-site.com/\", waiting until "domcontentloaded"\n"

Is this reproducible?

Yes

Inputs Causing the Bug

Steps to Reproduce

Code snippets

OS

Linux

Python version

3.13

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

Lenguaje dominante
Python
Estrellas
87.2k
Forks
9.1k
Merge medio
1 d 6 h
PR fusionados (30 d)
25

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de unclecode/crawl4ai

Todos los issues de unclecode/crawl4ai

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.