Scraper API
A service that takes a target URL and returns the page data, handling proxies, rendering, retries and anti-bot friction behind a single endpoint.
The product is one HTTP call. You send a target URL, plus options such as the country to appear from or whether to render JavaScript, and the response is the page, as HTML or parsed data. Everything this glossary spends pages explaining, choosing exits, rotating them, matching fingerprints to addresses, retrying failures, surviving challenges, happens inside the endpoint, run by people whose whole job is keeping it working.
It exists because the difficulty of collection is wildly uneven. Most sites need nothing: a plain request through a reasonable proxy succeeds, and always will. A minority sit behind defences that are re-tuned continuously, where keeping access is not a configuration you find once but a position you maintain, and the maintenance is somebody's ongoing working time. The API converts that staffing problem into a metered service: the operator maintains the position across all their customers at once, and you consume the result.
The billing model is the tell for what you are actually buying. Proxy plans meter traffic; scraper APIs typically charge per successful request, with failures free and harder configurations, JavaScript rendering, premium exits, priced as multipliers. Per-success pricing puts the retry risk on the operator, which is exactly the risk you were trying to shed. It also makes comparison honest: cost per thousand RESULTS, not cost per gigabyte moved, is the number that decides which approach is cheaper for a given target.
The trade is control and economics. On easy targets the API's premium buys nothing, because raw proxies already succeed there at a fraction of the price, and at very large volume the per-request pricing compounds against you. Inside the endpoint you also inherit its choices: its pacing, its session behaviour, its parsing, where your own client would be precisely tunable. So the sensible split runs by difficulty: raw proxies for the broad easy majority, the API for the hard tail where its operators' maintenance is the product, and a willingness to move targets between the two lanes as their defences change.
One capability distinction worth checking before integrating: some products return the raw rendered page and leave extraction to you, others return structured fields for specific site types, and the second kind quietly breaks differently, since a layout change becomes the operator's parsing bug rather than your fetch failing. Know which failure you would rather own.
How HProxy handles it
We run scraper APIs alongside the proxy families because the two lanes genuinely suit different targets, and we will say which fits yours: easy sites are cheaper on plain proxies, and paying our per-request rate there would be buying help you do not need.
Frequently asked questions
When should I use a scraper API instead of proxies?
Split by difficulty, not loyalty. Targets where plain requests through decent exits succeed are cheaper on raw proxies and stay that way at volume. Targets whose defences demand continuous adaptation are where per-success pricing and someone else's maintenance earn the premium. Many pipelines run both lanes at once and move targets between them as sites harden or relax.
Why are scraper APIs billed per request instead of per gigabyte?
Because the deliverable is an outcome rather than transport. Charging only for successful results moves retry and failure risk onto the operator, which is much of what you are paying to be rid of, and it prices difficulty honestly through multipliers for rendering or premium routing. It also gives you the right comparison number: cost per thousand results against your all-in proxy cost.
What does a scraper API actually do behind the endpoint?
The full stack this glossary describes, operated continuously: exit selection and rotation, session and cookie handling, header and fingerprint coherence, JavaScript rendering where asked, challenge handling, retries with different identities, and monitoring for the silent failure where a target starts serving degraded content. The value is less any single technique than the fact that somebody's job is keeping them current.
What are the downsides of scraper APIs?
Cost and control. On undefended targets the premium buys nothing, and at large scale per-request pricing can exceed a proxy-based pipeline severalfold. You also inherit the operator's choices on pacing, sessions and parsing, which a hand-built client would let you tune, and structured-output products move layout breakage from your code to their parsers, a failure you can no longer fix yourself.
Back to the full glossary.