Three Python libraries carry almost all proxied HTTP traffic: requests, httpx, and aiohttp. They look interchangeable in a one-line example and are not, and the differences that matter show up exactly where proxies live, in how each one is configured, whether it speaks SOCKS, and whether it can run a thousand requests at once. This guide compares the three for proxy work specifically, so you can pick the right one for a scraper instead of porting your code twice.
We run a proxy network and a live proxy checker, so we see all three in the wild, along with the same wrong turns: a requests script that should have been async, an aiohttp rewrite that kept the requests proxy pattern and silently used the real IP, an httpx job with no SOCKS extra installed. Each library has one shape it is best at, and the point of this article is to name it. Every example is runnable, and where one needs a live proxy, our free proxy API returns fresh endpoints with no key.
The one-paragraph answer
Use requests for a simple synchronous script where clarity beats everything. Use httpx when you want the same simple API but with HTTP/2, stricter defaults, and the option to go async later without changing libraries. Use aiohttp (or async httpx) when the job is thousands of requests through a rotating pool, because that work is IO bound and only concurrency makes it fast. The choice is really sync versus async, and the library follows from it.
How each one configures a proxy
This is where the three diverge most, and where ported code breaks.
requests takes a proxies dict keyed by the scheme of the target URL:
proxies = {"http": "http://user:pass@host:8080", "https": "http://user:pass@host:8080"}
requests.get(url, proxies=proxies, timeout=(5, 15))
httpx takes a single proxy for the whole client, or a mounts map for finer control, set on the client rather than as a target-scheme dict:
import httpx
with httpx.Client(proxy="http://user:pass@host:8080", timeout=15) as client:
client.get(url)
aiohttp takes the proxy as an argument on each request and has no client-level proxy at all:
async with aiohttp.ClientSession() as session:
async with session.get(url, proxy="http://user:pass@host:8080") as resp:
...
That last row is the one that catches people. Setting a proxy on the aiohttp session the way you would on a requests session does nothing, and every request quietly goes out on the real IP.
Config style
requests
proxies dict, keyed by target scheme
httpx
proxy= or mounts= on the Client
aiohttp
proxy= on each request, no client setting
Reads env by default?
requests
yes, trust_env on
httpx
yes, trust_env on
aiohttp
no, opt in with trust_env=True
SOCKS support
All three can speak SOCKS5, but each needs a different extra, and none ships it by default:
| Library | Install | Proxy value |
|---|---|---|
| requests | pip install 'requests[socks]' | socks5h://host:1080 in the proxies dict |
| httpx | pip install 'httpx[socks]' | socks5://host:1080 as the proxy |
| aiohttp | pip install aiohttp_socks | ProxyConnector.from_url('socks5://...') |
The socks5 versus socks5h distinction is the same in all three and worth getting right: the plain form resolves the hostname on your machine, so your local resolver sees every target, while the h form (or aiohttp's rdns=True) resolves on the proxy. For scraping and privacy the proxy-side lookup is almost always what you want. We cover it in depth in what is a SOCKS5 proxy.
Sync, async, and why it decides everything
requests is synchronous only. Each request blocks until it returns, so a thousand URLs run one after another, and through a proxy that means a thousand connection setups in series. For a handful of requests that is fine and the simplest thing that works.
httpx is synchronous and asynchronous from one library: httpx.Client mirrors requests, and httpx.AsyncClient runs under asyncio. aiohttp is asynchronous only and built for it from the ground up. Once a job is large enough that the exit IP should change per request, async is the difference between a crawl that finishes in minutes and one that finishes overnight, because the bottleneck is waiting on the network, not the CPU, and async lets hundreds of those waits overlap.
The rule of thumb: if you are rotating a pool across many requests, you want async, which means aiohttp or httpx.AsyncClient. If you are making a few calls in a script, sync is clearer and requests or httpx.Client is the right reach. We work the async pattern end to end, with the semaphore and the per-request rotation, in the aiohttp guide.
Connection reuse
The expensive part of a proxied request is the connection setup, especially the tunnel to the proxy, so reuse matters more with a proxy than without one. All three pool connections when you keep the client alive: a requests.Session, an httpx.Client, or an aiohttp.ClientSession. The mistake is the same in every library, calling the top-level get in a loop and re-opening the tunnel every time. Create one client, reuse it, and close it when done. httpx and aiohttp both push you toward this with a context manager; requests lets you forget, so it is the one where a stray requests.get in a loop quietly costs you a handshake per call.
Which to pick
- A quick script, a few requests, readability first: requests. It is the most widely known and the examples are everywhere.
- A new project you want on modern footing: httpx. Same feel as requests, plus HTTP/2, enforced timeouts by default, and a clean path to async when you need it, all without a second library.
- A high-volume scraper rotating a pool: aiohttp or
httpx.AsyncClient. The concurrency is the whole point, and both are built for it.
None of them fixes the parts a proxy library cannot: verifying each exit before you trust it, and pulling from a pool that is actually alive. For the first, our proxy checker reports exit IP, anonymity grade, geolocation and latency in one paste. For the second, the free proxy API hands you recently checked endpoints with no key. When a hand-rolled rotation loop outgrows a list, a rotating gateway that returns a fresh residential IP per request at $0.44/GB removes the list management for good. From here, the per-library guides go deep: requests, httpx, and aiohttp, and proxies for web scraping covers the request hygiene all three still need.