How to Use Proxies With cURL: Flags, Gotchas and Copy-Paste Recipes

Using a proxy with curl is where most people meet proxies for the first time: mid-debugging session, copying a -x flag off a forum without knowing what it does. This guide is the reference we wanted that day: every proxy-related flag that matters, the three gotchas that eat an afternoon each, and tested recipes for working with whole proxy lists instead of single addresses.

Everything below is copy-paste runnable. Where an example needs a live proxy, we pull one from our free proxy API, which returns real, recently verified endpoints without a key.

How do you use a proxy with curl?

Pass the proxy to curl with the -x flag: curl -x http://host:port https://example.com. Add -U user:pass for authentication, switch the scheme to socks5h:// for SOCKS5 with remote DNS, or set the lowercase http_proxy variable to proxy every request in a script. That one flag covers almost everything below.

The basic flag: -x

One flag does almost everything:

curl -x http://203.0.113.7:8080 https://example.com/

-x (long form --proxy) takes a proxy URL: scheme, host, port. If you omit the scheme, curl assumes an HTTP proxy. If you omit the port, it assumes 1080, a SOCKS default that surprises people who expected 8080, so just always write the port.

The scheme is where the real decisions live:

Two of these hide traps worth spelling out.

Gotcha 1: socks5 vs socks5h

With socks5://, your machine performs the DNS lookup and sends the proxy an IP address. Your local resolver (and network) sees every hostname you visit, and if the name only resolves inside the proxy's network, the request fails entirely. With socks5h://, the hostname travels to the proxy and resolution happens there.

# DNS resolved locally: your resolver sees "httpbin.org"
curl -x socks5://198.51.100.14:1080 https://httpbin.org/ip

# DNS resolved by the proxy: your resolver sees nothing
curl -x socks5h://198.51.100.14:1080 https://httpbin.org/ip

If you use SOCKS5 for privacy, socks5h is almost always what you meant. We covered why in the SOCKS5 deep dive.

Gotcha 2: https:// proxy scheme vs proxying HTTPS URLs

-x https://proxy:port does not mean "proxy my HTTPS traffic." It means curl speaks TLS to the proxy, which very few proxies (mostly modern commercial gateways) support. A normal HTTP proxy carries your HTTPS traffic just fine through a CONNECT tunnel, so -x http://proxy:port https://target is the standard, correct combination even though the schemes look mismatched.

curl proxy authentication

Paid proxies authenticate with username and password. Two equivalent spellings:

# -U / --proxy-user
curl -x http://gate.example.com:8000 -U username:password https://httpbin.org/ip

# Credentials inline in the proxy URL
curl -x http://username:[email protected]:8000 https://httpbin.org/ip

Prefer -U in scripts: inline credentials end up in shell history and process lists, and characters like @ or : inside the password break URL parsing unless percent-encoded. If the proxy rejects the credentials you get HTTP 407 Proxy Authentication Required, which is your cue that the proxy is alive and the credentials are the problem.

Environment variables, and the uppercase trap

curl (like most Unix tooling) honors proxy environment variables, which is how you proxy a whole script without touching each command:

export http_proxy="http://203.0.113.7:8080"
export https_proxy="http://203.0.113.7:8080"
export no_proxy="localhost,127.0.0.1"
curl https://httpbin.org/ip   # now proxied, no -x needed

The trap: for plain-HTTP requests, curl reads only the lowercase http_proxy. The uppercase HTTP_PROXY is ignored on purpose, because CGI servers copy the client's Proxy: request header into HTTP_PROXY, which would let strangers choose your proxy (this class of bug got the name "httpoxy"). The other variables work in both cases, but the habit that never fails is: lowercase, always.

no_proxy takes a comma-separated list of hosts that bypass the proxy; the one-off flag version is --noproxy "localhost,127.0.0.1". And when an environment variable is proxying you without your consent (a surprisingly common CI mystery), curl -v shows the proxy in the connect line, and --noproxy "*" turns it all off for one command.

Seeing what the target sees

A proxy is only doing its job if the target sees its address, not yours. Verify, never assume:

# Your IP without the proxy
curl -s https://httpbin.org/ip

# Through the proxy: should print the proxy's exit IP
curl -x http://203.0.113.7:8080 -s https://httpbin.org/ip

# What headers arrive at the target (watch for X-Forwarded-For / Via)
curl -x http://203.0.113.7:8080 -s https://httpbin.org/headers

If X-Forwarded-For in that last output contains your real IP, the proxy is transparent grade and hides nothing. Our proxy checker runs this whole battery (exit IP, anonymity grade, real exit geolocation, latency) in one paste if you would rather not script it.

Timeouts, timing and retries

Free and overloaded proxies hang more often than they refuse, so give every scripted request a budget:

curl -x http://203.0.113.7:8080 \
     --connect-timeout 5 \
     --max-time 15 \
     --retry 2 --retry-connrefused \
     -s https://httpbin.org/ip

--connect-timeout caps the handshake, --max-time caps the whole transfer, and --retry re-attempts transient failures. For measuring a proxy instead of just using it, -w prints timing splits:

curl -x http://203.0.113.7:8080 -o /dev/null -s \
     -w "connect: %{time_connect}s  ttfb: %{time_starttransfer}s  total: %{time_total}s\n" \
     https://example.com/

time_connect isolates "how far away and how loaded is this proxy," which is the number our own verification engine cares most about when it grades latency.

Recipes for whole lists

Single proxies are for debugging; real work uses pools. These three recipes cover most of it.

Fetch fresh proxies programmatically. Our free API returns the live pool in plain text, JSON or CSV, no key required:

# 20 fresh HTTP proxies, one ip:port per line
curl -s "https://hproxy.com/api/proxy-list?format=txt&protocol=http&recent=true&limit=20"

Test a whole list in parallel, keep the survivors. Feed any list (that API, or a file) through xargs:

curl -s "https://hproxy.com/api/proxy-list?format=txt&recent=true&limit=100" |
xargs -P 10 -I{} sh -c \
  'curl -x http://{} --connect-timeout 5 --max-time 10 -s -o /dev/null \
        -w "%{http_code} {}\n" https://httpbin.org/ip' |
grep '^200' | awk '{print $2}' > working.txt

Ten parallel workers, five-second connection budget, and working.txt ends up holding only proxies that completed a real request end to end. Expect heavy attrition on any free list; that is the nature of the material, as we laid out in our take on free proxies.

Rotate per request. Simplest possible rotation, no tooling:

mapfile -t PROXIES < working.txt
for url in $(cat urls.txt); do
  p=${PROXIES[RANDOM % ${#PROXIES[@]}]}
  curl -x "http://$p" --max-time 15 -s "$url" -o "out/$(basename "$url").html"
done

For production scraping you would move to a gateway that rotates server-side (one endpoint, fresh residential IP per request, no list management at all), but the loop above is unbeatable for understanding what rotation actually does.

Common curl proxy errors, by message

The meta-rule for all of these: establish whether the failure is proxy-side or target-side by swapping exactly one variable at a time. Same request, no proxy: does it work? Same proxy, boring target like httpbin: does it work? Two commands, and the mystery is gone.

Where to go from here

Keep a fresh pool within reach: the free proxy list re-verifies its entries every few minutes. Verify anonymity with the checker before you trust any proxy with real traffic. And when a project graduates from experiments to production, get IPs nobody else is burning, which is the whole pitch for paid pools.