Owner

scrapy-plugins

4 indexed repositories · View on GitHub

scrapy-playwright
🎭 Playwright integration for Scrapy

Python · 1447 stars
scrapy-zyte-smartproxy
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

Python · 363 stars
scrapy-zyte-api
Zyte API integration for Scrapy

Python · 43 stars
scrapy-download-handlers-incubator
Additional download handlers for Scrapy

Python · 3 stars

Beginner friendly Everything

Open All

Question: Still need ScrapyCamoufoxDownloadHandler with new Pluggable Browser / Camoufox? Open

question

Difficulty 4/5 3-5 days Newbie friendliness 25/100

scrapy-plugins/scrapy-playwright#386 · 13 comments · Opened Jul 20, 2026
Proxy certificate checking with custom bundle support Open

cross-handler enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-download-handlers-incubator#60 · 2 comments · Opened Jul 1, 2026
Support automatic big download header use Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-api#305 · Opened Jun 19, 2026
websockets download handler Open

new handler

Difficulty 5/5 Over a week Newbie friendliness 35/100

scrapy-plugins/scrapy-download-handlers-incubator#28 · Opened Apr 17, 2026
aiohttp custom DNS resolvers/settings Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 45/100

scrapy-plugins/scrapy-download-handlers-incubator#27 · Opened Apr 17, 2026
Impersonation settings for curl_cffi Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 15/100

scrapy-plugins/scrapy-download-handlers-incubator#26 · Opened Apr 17, 2026
Twisted-based handler with the new flow Open

new handler

Difficulty 5/5 Over a week Newbie friendliness 20/100

scrapy-plugins/scrapy-download-handlers-incubator#25 · Opened Apr 17, 2026
curl_cffi handler without stream=True Open

new handler

Difficulty 3/5 1-2 days Newbie friendliness 38/100

scrapy-plugins/scrapy-download-handlers-incubator#24 · Opened Apr 17, 2026
HTTP/3 support Open

cross-handler enhancement

Difficulty 5/5 Over a week Newbie friendliness 20/100

scrapy-plugins/scrapy-download-handlers-incubator#23 · Opened Apr 17, 2026
Flaky tests since 0.33.0 Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

scrapy-plugins/scrapy-zyte-api#277 · 1 reaction · Opened Apr 3, 2026
Let page objects request arbitrary Zyte API params Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-api#264 · Opened Sep 4, 2025
Switch spider middlewares to BaseSpiderMiddleware Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-api#257 · Opened May 12, 2025
Chrome extension dont load correctly on scrapy-playwright Open

Stale

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-playwright#342 · 3 comments · Opened Apr 2, 2025
Do not require ZYTE_SMARTPROXY_APIKEY for Zyte API Open

enhancement help wanted

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-smartproxy#136 · 1 comment · Opened Apr 1, 2025
Zyte API cookies are no longer experimental Open

Difficulty 3/5 1-2 days Newbie friendliness 52/100

scrapy-plugins/scrapy-zyte-api#243 · Opened Jan 27, 2025
Tests take too long Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-api#235 · 1 comment · 1 reaction · Opened Dec 6, 2024
Unnecessary base64 decoding Open

Difficulty 2/5 1-3 hours Newbie friendliness 55/100

scrapy-plugins/scrapy-zyte-api#230 · Opened Oct 31, 2024
DummyResponse Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-api#226 · 5 comments · Opened Oct 14, 2024
Firefox does not work with proxy. Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

scrapy-plugins/scrapy-playwright#320 · 6 comments · Opened Sep 20, 2024
Unexpected termination of scrapy-playwrigth jobs; caused by node:events:496 Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

scrapy-plugins/scrapy-playwright#316 · 6 comments · Opened Sep 3, 2024
Scrapy Playwright load chrome extensions and configure them Open

support

Difficulty 5/5 Over a week Newbie friendliness 20/100

scrapy-plugins/scrapy-playwright#310 · 4 comments · 2 reactions · Opened Aug 7, 2024
Document the need to add 520 and 521 to RETRY_HTTP_CODES when using Zyte API proxy mode Open

docs enhancement

Difficulty 3/5 1-2 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-smartproxy#122 · Opened May 28, 2024
Track retries in the crawler's stats Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

scrapy-plugins/scrapy-zyte-api#186 · 2 reactions · Opened Apr 8, 2024
After v2.3.0 I'm getting timeouts to URLs Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

scrapy-plugins/scrapy-zyte-smartproxy#113 · 2 comments · Opened Nov 23, 2023
Cannot download binary file (PDF) with Chromium headless=new mode Open

upstream issue

Difficulty 4/5 3-5 days Newbie friendliness 25/100

scrapy-plugins/scrapy-playwright#243 · 13 comments · Opened Nov 15, 2023
Guideline on how to use scrapy-playwright based on a real corporative use case Open

documentation

Difficulty 4/5 3-5 days Newbie friendliness 25/100

scrapy-plugins/scrapy-playwright#242 · 3 comments · 4 reactions · Opened Nov 3, 2023
create facilities or examples for retrying of browser automation failure. Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-api#120 · 1 reaction · Opened Aug 7, 2023
Raise error on unsuccesful automation Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

scrapy-plugins/scrapy-zyte-api#119 · 1 comment · Opened Aug 7, 2023
Template cache policy for failed browser automation Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-api#118 · 1 comment · Opened Aug 7, 2023
ZyteApiProvider could make an unneeded API request Open

scrapy-plugins/scrapy-zyte-api#91 · 16 comments · 1 assignee · Opened Jun 12, 2023
images don't appear to get read from the persistent context properly / cached Open

upstream issue

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-playwright#198 · 3 comments · Opened May 10, 2023
ERROR: Task was destroyed but it is pending! Open

upstream issue

Difficulty 4/5 3-5 days Newbie friendliness 25/100

scrapy-plugins/scrapy-playwright#188 · 10 comments · 1 reaction · Opened Mar 31, 2023
Page hangs on function instead of redirecting Open

needs more info

Difficulty 4/5 3-5 days Newbie friendliness 25/100

scrapy-plugins/scrapy-playwright#147 · Opened Dec 10, 2022
Scrapy-palywright cannot start working if the reactor is already installed Open

needs more info

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-playwright#131 · 12 comments · 1 reaction · Opened Oct 11, 2022
Use text response objects for text responses without headers Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 42/100

scrapy-plugins/scrapy-zyte-api#47 · 1 comment · 1 reaction · Opened Aug 29, 2022
Proxy removes cookies Open

bug upstream issue

Difficulty 3/5 1-2 days Newbie friendliness 35/100

scrapy-plugins/scrapy-playwright#117 · 5 comments · Opened Aug 9, 2022
Does not shutdown cleanly on SIGINT (cmd+c) Open

upstream issue

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-playwright#62 · 2 comments · 1 reaction · Opened Feb 23, 2022
Blacklist domains Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-smartproxy#94 · 1 comment · 1 reaction · Opened Feb 10, 2021
breaks `response.text` for other middlewares Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-smartproxy#93 · 1 comment · Opened Jan 27, 2021
Support more secure ways to declare the APIKEY Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

scrapy-plugins/scrapy-zyte-smartproxy#88 · 1 comment · Opened Jul 28, 2020
Native support for X-Crawlera-Session Open

enhancement help wanted

Difficulty 4/5 3-5 days Newbie friendliness 35/100

scrapy-plugins/scrapy-zyte-smartproxy#27 · 11 comments · Opened Oct 29, 2016

scrapy-plugins

Get new issues in your inbox