The hosted Firecrawl API brings its own proxies. A self-hosted server brings none. It has no stealth engine, and every request leaves from your server's IP until you set PROXY_SERVER. That one setting, with PROXY_USERNAME and PROXY_PASSWORD, now reaches the browser, the plain fetch engine and the search fallback. We read every setting on this page in Firecrawl's code at release v2.11.0, of 19 June 2026, and on the main branch as of 19 September 2026. The compose file builds whatever you cloned, so both count.
Why does a self-hosted Firecrawl get blocked?
On a rented server, the address is the first thing many sites judge. Firecrawl's own browser service says so when it starts a scrape without a proxy: "WARNING: No proxy server provided. Your IP address may be blocked."
We tested this on 19 September 2026. The same plain request that Firecrawl's fetch engine sends went to thirteen sites, twice from our server in a hosting network and twice through a residential line. Four of the answers:
| Site | From our server IP | From a residential IP |
|---|---|---|
| DuckDuckGo HTML search | failed the connection 3 times in 5 | results, 2 times in 2 |
| Zillow listings | 403, "Access to this page has been denied" | the listings page |
| 403, "blocked by network security" | a script check, no posts | |
| Indeed job search | 403 with Cloudflare's challenge | the same 403 |
The DuckDuckGo row matters most here. It is the exact page a self-hosted /search reads when it has no SearXNG. In all, the residential IP changed the answer at four of the thirteen sites, and four refused both IPs. The full table is on our OpenClaw page.
Where does the proxy go?
Into the .env file next to docker-compose.yaml:
# .env
PROXY_SERVER=http://GATEWAY_HOST:GATEWAY_PORT
PROXY_USERNAME=USERNAME
PROXY_PASSWORD=PASSWORD
Then restart the stack with docker compose down and docker compose up -d. PROXY_SERVER takes a full URL or a bare host:port, which counts as http://. Leave the two login lines empty when the proxy knows your server by its IP.
The compose file already passes all three settings to the API service, which also runs the workers, and to the Playwright service. Guides that tell you to add them to the browser service by hand describe an older compose file. The fetch engine ignored the proxy until issue #1035 was fixed in February 2025.
Which parts of Firecrawl use the proxy?
The .env.example still calls the three lines "Proxy Settings for Playwright". The code uses them in more places:
| Part | Uses PROXY_SERVER | How |
|---|---|---|
| Browser, through the Playwright service | yes | a proxy for each browser context |
| Fetch engine, plain requests | yes | the proxy client, login sent as a header |
/search without SearXNG | yes | DuckDuckGo's HTML page, through the fetch client |
/search with SearXNG | no | SearXNG sends the searches from its own host |
| Stealth proxies of the hosted API | not present | they live in fire-engine, which runs only there |
On the main branch, the browser no longer talks to your proxy directly. The Playwright service starts a small checking proxy on 127.0.0.1, which forwards each allowed request to PROXY_SERVER. The settings you write stay the same.
Which proxy URL should I write?
Use an http:// line when the proxy needs a login. Three facts decide this. Playwright documents its username and password for HTTP proxies. Chromium, which the browser service runs, supports no SOCKS5 login. And the fetch engine sends the login as an HTTP header, which a SOCKS5 proxy does not read.
Set both login lines, never one. In release v2.11.0, the browser gets the login only when PROXY_USERNAME and PROXY_PASSWORD are both set. With one missing, the browser does not log in at all.
What does stealth mode do on a self-hosted server?
Nothing you can use. On the hosted API, proxy: "stealth" and "enhanced" pick engines with stealth proxies. Those engines belong to fire-engine, which a self-hosted server does not run.
The default mode, "auto", still reacts to blocks. When a page answers 401, 403 or 429, Firecrawl retries it with the stealth flag. On your own server that retry finds no stealth engine. In a case traced on GitHub in 2025, issue #2056, it fell through to the PDF and document engines instead. When every engine fails, the error reads "All scraping engines failed to retrieve content from this URL". The fix is the IP in PROXY_SERVER, not the mode.
What breaks when the proxy is on?
- "Connection violated security rules." The fetch engine closes any connection whose other end is a private address, and with a proxy that other end is the proxy. A forwarder on a private address, for example in the same Docker network, trips it. The switch that allows it,
ALLOW_LOCAL_WEBHOOKS, is not passed to the API services by the compose file. PointPROXY_SERVERat the proxy's public address. We read this in the code and did not run it. - "DNS lookup failed ... cannot verify target is safe". On main the message says the target "resolves to a private/internal address". The browser service looks up every site itself before the proxy sees it. On a host that reaches the web only through the proxy, that lookup fails and every page is refused. Issue #4349 reports this and is still open.
ALLOW_LOCAL_WEBHOOKS=trueturns the check off, but it also lets Firecrawl open addresses inside your network. - 407 Proxy Authentication Required. The proxy refused the login. On our gateways this means a wrong password or a line from another plan. Our 407 guide walks through it.
- The browser is refused while plain pages work. One of the two login lines is empty, so only the fetch engine logs in. Set both.
- A setting in your compose file does nothing. Firecrawl reads exactly three proxy settings. One guide sets a rotation URL that no Firecrawl code reads.
- The proxy bill grows fast. A browser loads every image and video on the page. Set
BLOCK_MEDIA=true, which the.env.exampleoffers "to save proxy bandwidth". In our bandwidth study, a headless browser pulled 6.54 MB per protected page.
Which proxy type fits a self-hosted Firecrawl?
Residential, for the sites that refuse a hosting network. Our test shows what that buys and what it does not. It lifted the refusals that judged only the IP. It did not pass Cloudflare's script check with a plain request, so those sites need the browser engine as well, which we did not test.
Firecrawl takes one proxy for everything. A rotating line spreads the fetch engine and search over many IPs, since each request stands alone. If a site needs one IP for a whole visit, point PROXY_SERVER at a sticky port instead. Every request then shares that IP until the session ends.
HProxy residential gateways fit both. Allow your server's IP on a Residential Premium plan, up to 150 per plan, and leave the login lines empty. That also avoids the SOCKS5 and header limits above. An allowed IP takes no country or city targeting. When a job needs a country, use a generated line with the targeting in its username, as an http:// line with both login settings. The residential proxies page lists the plans, and the plan API generates lines and manages allowed IPs from code.
What this page does not cover
We read Firecrawl's code at release v2.11.0 and on main, and we did not run Firecrawl. The test used plain requests from one server IP and one residential line, over one afternoon. It did not run Firecrawl's engines or a browser behind the residential IP. The private-address refusal and the stealth retry come from reading the code and one traced issue, not from a run. The chart describes release v2.11.0: we did not check SOCKS5 lines through the checking proxy on main. Firecrawl's main branch moves daily, so we will read these settings again by 19 October 2026.
Where to go from here
Proxies for Hermes Agent covers an agent that uses Firecrawl as its web backend. The Playwright guide covers the browser library behind Firecrawl's browser service. HTTP vs SOCKS5 explains the two proxy kinds. If you are weighing the hosted API against your own server, our Firecrawl pricing breakdown covers the hosted plans.
Sources
- SELF_HOST.md, docker-compose.yaml and apps/api/.env.example. Firecrawl, release v2.11.0, 19 June 2026, and main at 719efcde77a9, 19 September 2026.
- config.ts, safeFetch.ts, engines/index.ts, scrapeURL/index.ts, error.ts, v2 types, search/v2 and the Playwright service. Firecrawl source code, release v2.11.0 and main, read on 19 September 2026.
- The proxy option. Playwright documentation, version 1.58.1.
- net/docs/proxy.md, the SOCKS section. The Chromium Authors, read on 19 September 2026.
- ProxyAgent. undici source code, version 7.24.1.
- Issues #1035, #1129, #925, #680, #4349, #2056 and #826. Firecrawl issue tracker, 2024 to 2026.
- Plans, IP whitelist and sticky sessions; errors; the proxy API. HProxy documentation, hproxy.com/docs, 19 September 2026.
- Headless browsers and proxy bandwidth. HProxy data study, 6 August 2026, updated 1 September 2026.
- Our own test of 19 September 2026: plain GET requests to 13 sites and 3 controls, two runs from our server and two through a residential line of our own house plan, with curl 8.5.0. Raw output is kept in the research folder of our OpenClaw page.
