Use case

Proxies for DeerFlow: one tool takes an address, the other does not

Set a proxy in DeerFlow 2.0: the web_fetch proxy and trust_env keys, what to do about the search tool, and what must stay direct in Docker.

HProxy Team··Updated September 20, 2026·6 min read
HProxy.Use case

Free proxies won't hold up here.

Shared datacenter IPs get flagged and dropped fast. When it has to hold, gaming, streaming, accounts, you need mobile and residential IPs that read as a real device, from $0.44/GB, pay as you go.

Proxies for AI Agents

A research agent is mostly a machine for reading web pages. DeerFlow reads a lot of them, and when it runs on a server or behind a company network, some of that reading stops working. The fix is not where most people look: there is no global proxy setting in this project, and the two halves of the job, searching and fetching, answer the question differently. We read release 2.0.0 on 20 September 2026.

What a run actually fetches

The agent ships twelve integrations. Most are commercial services reached with a key: search APIs, reader APIs, a browser service. For those your address rarely decides the answer. The key and the bill do. Two kinds of call are different: the page fetch, which reads the open web on your behalf, and the default search, which reads a search engine directly.

Those two are what a proxy is for, and only one of them has a setting.

Which tool takes a proxy

ToolProxy setting
web_fetch, default provideryes: proxy, trust_env, timeout
web_fetch, other providersnone, and only one provider can be active
web_search, default providernone in the config; the library reads its own variable
web_search, keyed servicesnot applicable, they take a key
web_search via a self-hosted enginethe engine's own proxy settings, not DeerFlow's

The fetch tool

tools:
  - name: web_fetch
    group: web
    use: deerflow.community.jina_ai.tools:web_fetch_tool
    timeout: 10
    proxy: $HTTPS_PROXY
    trust_env: true

Those three keys are read from the tool's own entry and handed to the HTTP client. The behaviour is sensible. An explicit address wins. Leave it out and the client still reads the standard environment variables, because the trust setting defaults to on. Setting that to false is how you keep this one tool off a system-wide proxy.

Two details from the project's own example are worth copying. The value can interpolate an environment variable, so the address does not have to sit in the configuration file. And if your proxy runs on the machine hosting your containers, use the Docker host name rather than a loopback address, because inside a container the loopback is the container.

One limitation follows from the shape. Only one fetch provider can be active at a time, and only the default one carries these keys. Switch the fetch tool to a browser service or another reader and the proxy setting goes with it.

The search tool, and the variable that saves it

The default search is built with a timeout and nothing else. Its configuration accepts a result count, a region, a safe search level and a backend. There is no proxy key, so a search that gets blocked cannot be fixed in the configuration file.

It can be fixed anyway. The library behind that search falls back to its own environment variable when the caller passes nothing:

export DDGS_PROXY="http://USER:PASS@GATEWAY_HOST:GATEWAY_PORT"

It accepts a login in the address and understands SOCKS as well as plain HTTP. It also has a shorthand that expands to a local Tor address, which tells you something about the audience the library expects.

This matters more than it sounds. In our own paired test, a residential address changed 4 of 13 answers, and one of those four was the search engine this agent reads by default: from our server its HTML search failed, and through a residential line it answered.

Containers, and what must stay direct

The container setup takes the proxy variables from your environment file rather than declaring them in the compose file. That is deliberate, and the reason bites elsewhere too. Listing them as bare keys in the compose file resolves them to nothing when the host shell has them unset. That nothing then overrides the value from your environment file. The project measured it and avoided it.

What the compose file does declare is the exclusion list, extended with the agent's own service names and the Docker host name. Internal calls between its services must not go through an external proxy, or the agent breaks itself. The same rule caught the local sandbox a month later, and was fixed the same way.

So when you set a proxy here, set the exclusions too.

Which proxy type fits it?

Residential, on the fetch tool, because that is the part reading pages that refuse a server address. Our test above is the size of that effect: four of thirteen answers changed, and four sites refused both addresses, which no proxy fixes.

Traffic follows the reading. A research run spends its bytes where it fetches pages, not where it thinks. So a per gigabyte line pays for the fetch tool and nothing else. That makes the cost easy to predict, and it makes the single-provider limit above worth a thought before you switch fetchers.

A sticky port suits a run that reads several pages from one site in sequence. Write a retry anyway: a sticky address can change early when its device leaves the network. The residential proxies page lists the plans and the plan API manages allowed addresses from code.

What breaks

  • The fetch tool times out and nothing else fails. That tool is the one behind the proxy question; check its three keys first.
  • The agent talks to itself through your proxy. Add its service names and the Docker host name to the exclusion list.
  • Your compose proxy silently disappears. Bare keys in the compose file overrode the environment file. Keep them in the environment file.
  • You switched fetch providers and lost the setting. Only the default provider has it.
  • The search is blocked and the config has no answer. Set the library's own variable.
  • A paid search API still refuses. That is a key or a quota, not an address.

What this page does not cover

We read the release as text and did not run the agent. So we did not measure a run's traffic, and we did not test the container path. The point that keyed services ignore your address is a general statement about API products rather than something we measured per provider. We read the default search and fetch tools closely, and the other ten integrations only by name and configuration entry. The repository is pushed daily and a 2.1 candidate is already tagged, so we will read the tool entries again by 20 October 2026.

Where to go from here

Proxies for SearXNG covers the self-hosted search engine you can point this agent at, which has the proxy settings its own search tool lacks. Proxies for Crawlee covers a scraper with the opposite design: one proxy strategy for everything. Proxies for Open WebUI covers the other self-hosted tool where a fetch and a model call leave by different routes.

Sources

  • The fetch tool and its three settings, in the backend package under community/jina_ai: tools.py and jina_client.py. The search tool's configuration, under community/ddg_search: tools.py. The self-hosted engine client, under community/searxng. The documented tool entries in config.example.yaml and in the backend configuration guide. bytedance/deer-flow, release v2.0.0 of 25 June 2026, read 20 September 2026.
  • Pull request 3430 and issue 3418, which added the fetch proxy and explain why it is per tool, including the compose behaviour they measured. Pull request 4444 on exempting local sandbox traffic. DeerFlow repository, read 20 September 2026.
  • The search library's own constructor and HTTP client, which read its environment variable. deedy5/ddgs, read 20 September 2026.
  • Our paired address test of 19 September 2026: 16 URLs, plain requests, two runs from our server and two through a residential line of our house plan. Raw output is kept in the research folder of our OpenClaw page.
  • Plans, allowed addresses and sticky sessions. HProxy documentation, hproxy.com/docs, 20 September 2026.

Frequently asked questions

Does DeerFlow have a proxy setting?
Not a global one. The page fetch tool has its own proxy, trust_env and timeout keys in its entry in config.yaml. Everything else follows the environment, or nothing.
How do I route the page fetch tool?
Add proxy and trust_env under the web_fetch entry in your configuration. The value can interpolate an environment variable, and if the proxy runs on the machine hosting your containers, use the Docker host name rather than a loopback address.
What about the search tool?
It has no proxy key. The default search builds its client with a timeout only. The library behind it falls back to its own DDGS_PROXY environment variable, which takes a login and a SOCKS address, so that is the lever.
What must stay direct in Docker?
Calls between the agent's own services and to its local sandbox. The project extends the no-proxy list with its internal service names, and later had to exempt sandbox traffic as well.
Do proxies help with the paid search APIs?
Rarely. Those are reached with a key and billed to your account, so your address is not usually what decides the answer. Route the tools that read the open web.

Proxies that don't die mid-job

Residential, ISP, datacenter and mobile, verified by the same engine that runs tens of millions of checks. They read as a real device and hold up under load. Pay as you go, and your balance never expires. $0.44/GB is the 2,000 GB+ rate; a single gigabyte is $0.50/GB, with no minimum order.

129M+ proxy checks run · 100+ countries · HTTP / HTTPS / SOCKS · re-checked every few minutes · no signup

HProxy.

Honest guides and comparisons on proxies, scraping and staying unblocked, from the team that runs the network.

RSS feed