Headless browser
A real browser running without a visible window, used to render JavaScript-heavy pages in automation.
A headless browser is a full browser, Chrome or Firefox, running without the graphical window a person would see. It renders pages exactly as a real browser does, executes their JavaScript and builds the same DOM, but it is driven by code rather than a human, which is what makes it the tool for scraping sites that a plain HTTP request cannot read.
The catch is that headless mode leaves traces. Historically the headless flag itself was detectable, and even now a headless instance tends to lack the plugins, the exact rendering quirks and the interaction patterns of a genuine user session, all of which fingerprinting can pick up.
So a headless browser is necessary for rendering but not sufficient for staying unnoticed. It is paired with a residential proxy to fix the network tell and with fingerprint and behaviour management to fix the rest.
The cost side is usually underestimated, and on per-gigabyte billing it is the thing that quietly wrecks a budget. A browser fetches the page the way a browser does: every image, font, stylesheet, tracking script and video preload. A product page whose useful data is fifty kilobytes of HTML can pull two or three megabytes in total. Multiply that across a large job on residential bandwidth and the browser, not the proxy, is what the invoice is for.
It is heavier in every other dimension too. A headless browser instance consumes hundreds of megabytes of memory and real CPU, where an HTTP request consumes almost nothing, so the concurrency you can sustain on the same machine drops by an order of magnitude. That constraint often matters more than the per-request speed difference.
Which makes the first question worth asking not how to run one well, but whether you need one at all. A great many pages that look JavaScript-rendered are fed by a JSON endpoint the page itself calls, and calling that endpoint directly returns the same data in a fraction of the size with no rendering. Finding it in the network tab is frequently the highest-value hour of a scraping project.
When you genuinely do need rendering, block what you will not parse. Every major automation framework can intercept requests by resource type, and refusing images, fonts, media and stylesheets cuts the bandwidth dramatically while leaving the DOM you actually came for intact.
Frequently asked questions
Do I need a headless browser to scrape?
Only when the data genuinely does not exist until JavaScript runs, which is less often than it appears. Check whether the page is fed by a JSON endpoint you can call directly, because that returns the same data far smaller and faster with no rendering at all. Reach for a browser when you have confirmed there is no simpler route.
How do I reduce bandwidth when using a headless browser?
Intercept requests and block the resource types you will not parse: images, media, fonts and usually stylesheets. Every major framework supports this. It commonly cuts page weight by an order of magnitude while leaving the DOM intact, which matters enormously on per-gigabyte residential billing.
Can websites detect a headless browser?
Yes, and treating it as an invisible real browser is the usual mistake. Beyond the historical headless flag, an instance tends to differ in the properties it exposes, the plugins and fonts present, rendering behaviour, and above all the complete absence of human interaction: no mouse movement, no scroll, no reading pauses. Behaviour gives it away more reliably than any single property.
Is a headless browser slower than HTTP requests?
Dramatically, in both time and resources. It downloads and renders an entire page rather than fetching a document, and each instance consumes hundreds of megabytes of memory and real CPU. The practical consequence is that the concurrency you can sustain on one machine drops by roughly an order of magnitude.
Back to the full glossary.