Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Performance regressions in generic scrapers

Aperta
  • Qualcosa non va?
#362 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
typescript
Ambito
backend, performance

Direzione di ricerca

Start by identifying the builds and scraper versions named in the issue, then inspect the changes between them and use the reported Snowflake APIFY_ANALYTICS comparisons to narrow down the regressions. Prioritize the playwright-scraper failure and restart spike, then investigate the cheerio-scraper memory spike and scaling-related web-scraper slowdown. Done means the regressions are root-caused and corrected, with the comparisons rerun to verify performance and reliability.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

t-universe

Summary

Build-over-build performance comparison across the scrapers in this repo (last 7 days, Snowflake APIFY_ANALYTICS, all-runs aggregate + a controlled 1-clean-item short-run subset). Three of the four scrapers show a real regression in the newer build; puppeteer-scraper looks like a clean improvement and is not part of the problem list below — see the note at the end.

cheerio-scraper — memory/efficiency regression

Builds: gTZUgxTMzuh0BBr4b (old) → 6EleDKkR2avUKBelE (new)

  • Peak memory on minimal (1-item output) runs more than doubled: 1,733 MB → 6,567 MB. The exact same peak value shows up in the full aggregate too, so this looks like one run spiking to 6.5 GB — worth pulling that run to check for a leak.
  • Avg CU per clean item worsened ~56% (avg items/run dropped 107.7 → 70.6, while avg CU/run stayed roughly flat).
  • Avg allocated memory +15%, avg CPU +26%.
  • Minor reliability dip: abort rate 0.1% → 0.4%, restarts introduced (0 → 0.05 avg/run).
  • Not all bad: median duration improved (10.7s → 6.6s).

web-scraper — slower and less efficient at scale

Builds: byk3boe85HewTmSD9 (old) → XHPrjngguiKJ17ZsN (new)

  • Avg duration +29.7%, p95 duration +35.5% in the full run mix; avg CU +21.9%.
  • CU per clean item worse by ~12% even after crediting the new build for producing slightly more items.
  • Reliability regressed: success rate -0.9pp, fail rate +0.3pp, timeout rate +0.5pp.
  • Regression is concentrated in heavier/larger runs — the controlled short-run (1-item) subset is nearly flat, so this isn't a fixed per-call overhead issue, it's something that scales with run size.
  • One improvement: peak memory down 32%.

playwright-scraper — severe regression, do not roll out further

Builds: 8Dwn9FInP9Z5gsRy1 (old) → vi1WR785wACojOJg7 (new)

  • Fail rate: 0.7% → 14.2% (+13.5pp) — roughly 1 in 7 runs now fail outright.
  • Avg restarts: 0.02 → 0.43 (~21.5×), consistent with a crash/retry loop.
  • Avg clean items/run collapsed 42% (24.1 → 13.9).
  • Even on minimal 1-item runs, avg duration is +35% and p95 is +47% slower.
  • CU per clean item worse by ~62%.
  • Recommend root-causing the failure spike before this build goes any further — this reads as a functional regression, not a tuning tradeoff.

puppeteer-scraper — looks good, please retest after the above are fixed

Builds: WGHufYdES3oOahj9x (old) → 9Ho7gB6bqt4AU1UtL (new)

  • Median duration nearly halved (72.4s → 36.1s), consistent across both the aggregate and controlled short-run subset.
  • CU per clean item slightly improved despite a higher raw avg-CU/run number (fully explained by producing ~25% more items/run).
  • Peak memory down ~20%, reliability unchanged.
  • No action needed here, but please re-run this comparison once the other three scrapers are patched, in case a shared dependency/base-image bump introduced in this cycle also affects puppeteer-scraper in ways not yet visible in the current traffic mix.

Generated from a 7-day build comparison (all-runs aggregate + 1-clean-item short-run subset) via Redash/Snowflake APIFY_ANALYTICS.

Lingua principale
TypeScript
Stelle
149
Fork
59
Merge medio
7h 19m
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apify/actor-scraper

Tutte le issue di apify/actor-scraper

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.