Glossary · Performance
Concurrency
How many requests you run at the same time, the main lever on scraping throughput and a common hidden limit.
Concurrency is the number of requests in flight at once. It, far more than any single request's speed, sets how much a scraper gets done: a job that fetches a hundred pages in parallel finishes in a fraction of the time of one that fetches them one after another, even if each individual request is no faster.
Proxies interact with concurrency in two ways. A pool lets you run high concurrency without piling requests onto one address, since each parallel request can take a different exit. But the proxy service itself may cap concurrent connections, and a plan that limits threads quietly limits throughput no matter how fast the network is.
The number worth checking on any proxy is therefore not just price per gigabyte but whether concurrent connections are limited. Unlimited concurrency is what lets a pool's real advantage, parallelism, actually show up.
How HProxy handles it
Our residential plans do not cap concurrent connections or threads, because a thread limit is a throughput limit wearing different words, and it is exactly the constraint that makes a cheap-looking plan slow in practice.
Back to the full glossary.