Scrapy
Everything tagged Scrapy.
5 articles
Topic Scrapy
Tutorial
Web scraping with Scrapy: spiders, pipelines and proxies
A Scrapy tutorial run end to end on a practice shop: a project, a spider, pipelines that drop bad rows, retries, speed settings, the HTTP cache and a proxy.
October 12, 2026 · 8 min read
Guide
Scrapy TunnelError: "Could Not Open CONNECT Tunnel With Proxy" (407, 403, 502) Explained
Scrapy's TunnelError means the proxy answered the CONNECT request with a status other than 200. What each status means, the Proxy-Authorization trap, and the fix.
August 24, 2026 · 6 min read
Free proxies
Free Proxies for Scrapy: What a Test of 300 Showed, and the Settings to Use
We sent the first request of a new Scrapy project through 300 free proxies from our list. 83 returned the file. The settings and middleware that follow.
August 23, 2026 · 10 min read
Guide
How to Use Proxies With Scrapy (Rotating, Retries, Auth)
Proxies in Scrapy 2.19 from its own docs and source: the meta key, credentials since the 2.6.2 fix, which download handler takes SOCKS5, rotation and retries.
July 21, 2026 · 10 min read
Use case
Proxies for Scrapy: how a spider takes a proxy, and what Scrapy does with it
How Scrapy takes a proxy, from the Scrapy 2.19 documentation: the meta key and the environment variables, credentials in the proxy URL, middleware order, what a retry does with the proxy, SOCKS5, and the settings that set the pace.
July 11, 2026 · 5 min read
Proxies that don't die mid-job
Residential, ISP, datacenter and mobile, verified by the same engine that runs tens of millions of checks. They read as a real device and hold up under load. Pay as you go, and your balance never expires. $0.44/GB is the 2,000 GB+ rate; a single gigabyte is $0.50/GB, with no minimum order.
129M+ proxy checks run · 100+ countries · HTTP / HTTPS / SOCKS · re-checked every few minutes · no signup
HProxy.
Honest guides and comparisons on proxies, scraping and staying unblocked, from the team that runs the network.