JavaScript rendering
The process by which pages build their content in the browser after loading, which decides whether fetching the HTML gets you data or an empty shell.
The request succeeds, the HTML arrives, and the data is not in it. View the same page in a browser and everything is there. Nothing malfunctioned: the site renders client-side, meaning the document you fetched is a nearly empty stage, and the content is built after load by JavaScript that fetches data and writes it into the page. Your HTTP client received exactly what the browser received; the browser then did work your client does not do.
Which architecture a site uses is the first diagnostic question of any scrape, and it is answerable in seconds. The browser's view-source shows the document as delivered; the developer tools' element inspector shows the tree after scripts ran. If the price is in the second and not the first, you are looking at client-side rendering. The network tab then shows the more useful fact: WHERE the scripts got the data, which is almost always a tidy JSON request your client could make directly.
That observation orders the whole toolbox. The best response to a rendered page is usually not to render: call the underlying data endpoint and receive structured fields at a fraction of the transfer. Nearly as good, server-rendered frameworks embed their full state as a JSON blob inside the delivered HTML, current-generation React and Next.js sites carry it in a script tag, and parsing that blob yields everything the page knows without executing anything. Only when neither exists does the heavy option earn its place: a headless browser that genuinely runs the page, with the cost and detectability that brings.
The middle case deserves its own note because it is now the common one. Hydration, the pattern where servers deliver real HTML and scripts then attach interactivity, means plenty of modern pages ARE scrapeable from the raw document, and the empty-shell era is partly over. The trap inverts: content present at fetch time can still be REPLACED after load, prices updated by a fresher request, regional variants swapped in, so matching what a user sees sometimes requires rendering even though the fetched HTML looked complete.
For proxy planning, rendering multiplies everything. A rendered page load issues dozens of requests, scripts, data calls, assets, each through your exit, so per-gigabyte billing feels rendering hardest, and the additional surface, real browser behaviour included, is exactly where detection systems look next. The decision is consequential enough to make per target, not per project: render the pages that require it, and fetch the data layer everywhere else.
Frequently asked questions
How do I tell if a page needs JavaScript rendering to scrape?
Compare the delivered document with the live tree: view-source against the element inspector. Data present only after scripts run means client-side rendering. Then open the network tab and find where the data came from; a JSON endpoint you can call directly usually appears, and that discovery routinely makes the rendering question moot.
Can I scrape a JavaScript site without a headless browser?
Very often. The page's own data endpoints can be called directly, and server-rendered frameworks embed their complete state as JSON inside the HTML, readable with one parse. A headless browser is the correct last resort for pages where data genuinely exists only after execution, not the default first move on anything that looks modern.
What is hydration and why does it matter for scraping?
Hydration is the pattern where the server delivers complete HTML and client scripts then attach interactivity on top. It matters because such pages ARE scrapeable from the fetched document, reversing the empty-shell assumption, with one caveat: scripts can still replace delivered content after load, so what you parsed and what a user currently sees can drift for data that updates live.
Does JavaScript rendering cost more bandwidth?
Substantially. Executing a page triggers the full cascade of script, data and asset requests through your exit, easily an order of magnitude beyond fetching the document alone, which per-gigabyte residential pricing turns directly into money. Blocking unneeded resource types recovers much of it, and skipping rendering in favour of the data layer recovers nearly all of it.
Back to the full glossary.