Scrapling takes a proxy on each of its three fetchers as proxy=, or a list through ProxyRotator. The HTTP Fetcher takes the login inside the URL. DynamicFetcher and StealthyFetcher take the same URL or a dict with server, username and password. Three things matter beyond the syntax. FetcherSession ignored its proxy before version 0.4.9. A failed proxy is written to the log with its password. And the MCP server can only get a proxy from the model. We read every setting on this page in Scrapling 0.4.15, released on 23 August 2026.
Why would Scrapling need a proxy?
A scraper runs into limits per IP. One user described it in Scrapling's issue tracker: "the website blocks me every 100-140 requests". We tested what the address alone changes, on 19 September 2026: plain requests to thirteen sites, twice from our server and twice through a residential line. A residential IP changed the answer at four of them: Zillow, Instagram, Reddit and DuckDuckGo. Indeed, Glassdoor, Amazon and Booking refused both. The table per site is on our OpenClaw page.
An IP is not the whole story. Scrapling's maintainer put it plainly in the same tracker: protections "will detect you over a given number of solutions if your fingerprint stays the same".
How do I set a proxy on each fetcher?
Each fetcher runs on a different engine, and takes the proxy in its own forms:
| Fetcher | Runs on | Proxy forms |
|---|---|---|
Fetcher, FetcherSession | curl_cffi | URL with the login, or URL plus proxy_auth=, or proxies= |
DynamicFetcher | Playwright | URL with the login, or a dict |
StealthyFetcher | Patchright | URL with the login, or a dict |
from scrapling.fetchers import Fetcher, StealthyFetcher
page = Fetcher.get(
"https://example.com",
proxy="http://GATEWAY_HOST:GATEWAY_PORT",
proxy_auth=("USERNAME", "PASSWORD"),
)
page = StealthyFetcher.fetch(
"https://example.com",
proxy={"server": "http://GATEWAY_HOST:GATEWAY_PORT", "username": "USERNAME", "password": "PASSWORD"},
block_webrtc=True,
)
For the browser fetchers, Scrapling splits a user:pass@ URL into Playwright's separate login fields itself, so both forms work. The dict also spares you URL-encoding a password with special characters. Two limits apply there. Only http, https, socks4 and socks5 are accepted, so a socks5h:// line stops with "Invalid proxy string!". And a SOCKS5 proxy with a login fails, because Playwright refuses it with "Browser does not support socks5 proxy authentication". Use the HTTP line of the same proxy.
Many guides still say StealthyFetcher runs on Camoufox. It did until version 0.3.13, when Patchright replaced it.
block_webrtc=True in the example is not decoration. It forces WebRTC to respect the proxy, so page scripts cannot read your machine's own IP address through it. It is off by default.
Sessions take the same proxy= for every request they send. Up to version 0.4.8, FetcherSession(proxy=...) was silently ignored and requests left from the real IP. The fix shipped in 0.4.9. On any version, fetch an IP echo page once through the session before a real run.
How does ProxyRotator work?
ProxyRotator takes a list and hands the session a different proxy for each request:
from scrapling.fetchers import FetcherSession, ProxyRotator
rotator = ProxyRotator([
"http://USERNAME:PASSWORD@GATEWAY_HOST:PORT_1",
"http://USERNAME:PASSWORD@GATEWAY_HOST:PORT_2",
])
with FetcherSession(proxy_rotator=rotator) as session:
page = session.get("https://example.com")
- Order. It cycles through the list in order. A custom strategy function can pick randomly, by weight or by any rule.
- One or the other. A session takes
proxy_rotator=orproxy=, never both. Aproxy=on a single request still overrides the rotator, for example a country line for one page. - Failures. When a request fails with an error that looks like a proxy failure, the retry takes the next proxy.
- Browsers. A browser sets its proxy per context, not per tab. So with a rotator, the browser fetchers open a separate context for each proxy, with one tab, and close both after the job. Nothing carries over between those contexts, a login included.
- Spiders. They treat 401, 403, 407, 429, 444, 500, 502, 503 and 504 as blocked and retry up to three times, each time with a fresh proxy.
Strings work with every session type. The dict form is for browser sessions. Rotation in browser sessions was broken in 0.4.2 and fixed in 0.4.4, so update if a rotator raises NotImplementedError.
Where does the proxy password show up?
A proxy login in Scrapling can end up in three places you might not expect:
- The log. A failed proxy is written as
Proxy '<proxy>' failed, with its full value. Scrapling's logger prints warnings by default. - The response. Every response keeps the proxy that fetched it in
response.meta["proxy"]. Scrapling's own docs example prints it. - The MCP server. It reads no proxy from its environment. The model passes
proxy=in its tool calls, so a URL with a login travels through the conversation.
On the HTTP fetcher, a URL without the login plus proxy_auth= keeps the password out of the log and the response. A dict does not, because it is logged whole. A proxy that allows your IP needs no password anywhere.
Which proxy type fits Scrapling?
Match it to the fetcher. Page-by-page reads with Fetcher suit a rotating line. A StealthyFetcher session that logs in should keep one IP, so a sticky line fits it. Browser fetchers load whole pages, and residential traffic is billed per gigabyte. Scrapling's docs say disable_resources made some sites about 25 % faster and "can help save your proxy usage". It is off by default, and so is block_ads.
HProxy residential gateways fit all three fetchers. On a server with a fixed IP, allow that IP on a Residential Premium plan, up to 150 per plan. The proxy URL then carries no password, so nothing sensitive reaches the log, response.meta or an MCP conversation. An allowed IP takes no country or city targeting. For a country, or for many identities, generate lines with the plan API: each sticky line holds its own IP, and ten of them make a ProxyRotator with ten stable identities. A sticky IP can still change early if its device leaves the network, so keep Scrapling's retries on. The residential proxies page lists the plans.
What breaks when the proxy is on?
- The session proxy does nothing. You run 0.4.8 or older. Update to 0.4.9 or newer.
- "Invalid proxy string!" A browser fetcher got a scheme other than http, https, socks4 or socks5.
- "Browser does not support socks5 proxy authentication". Use the HTTP line instead of SOCKS5.
NotImplementedErrorfrom a rotator in a browser session. You run a version older than 0.4.4. Update.- "Cannot use 'proxy_rotator' together with 'proxy' or 'proxies'". Pick one per session.
- A spider retries the same 407. The login is wrong, and every retry costs a request. On our gateways a 407 means a wrong password or a line from another plan. Our 407 guide walks through it.
- A leak test shows your own IP through WebRTC. Set
block_webrtc=TrueonStealthyFetcher. - Blocks come back after a while. The fingerprint stayed the same. A new IP does not change that.
What this page does not cover
We read Scrapling 0.4.15's docs, code and tests. We did not run Scrapling. The blocking test used plain requests from one server IP and one residential line, over one afternoon. We did not check how the HTTP fetcher handles a percent-encoded password, or a dict from a rotator. The optional Camoufox engine gets its own page. Scrapling releases every week or two, so we will read these settings again by 19 October 2026.
Where to go from here
Proxies for Playwright MCP covers the Playwright proxy rules behind the browser fetchers. Proxies for browser-use and proxies for self-hosted Firecrawl cover two other scraping tools. Sticky vs rotating sessions explains the choice above, and HTTP vs SOCKS5 the two proxy types.
Sources
- Fetchers basics, the HTTP fetcher, DynamicFetcher, StealthyFetcher, proxy management and handling blocks, and the MCP server. Scrapling documentation, D4Vinci/Scrapling release v0.4.15, 23 August 2026.
- construct_proxy_dict, ProxyRotator, the request engine and its log lines, the browser defaults, the MCP server and pyproject.toml. Scrapling source code, release v0.4.15.
- normalizeProxySettings. Playwright source code, release v1.62.0, the oldest Playwright that Scrapling 0.4.15 accepts.
- Issues #295, #215, #191, #127 and #24, pull requests #304 and #223, and the v0.4.4 release notes. Scrapling issue tracker, 2024 to 2026.
- Plans, IP whitelist and sticky sessions; errors; the proxy API. HProxy documentation, hproxy.com/docs, 19 September 2026.
- Our own test of 19 September 2026: plain GET requests to 13 sites and 3 controls, two runs from our server and two through a residential line of our own house plan, with curl 8.5.0. Raw output is kept in the research folder of our OpenClaw page.
