JavaScript web scraping: fetch, Cheerio and Puppeteer in Node.js

JavaScript can scrape the web with three tools that cover most jobs. Node's built-in fetch downloads a page, and Cheerio reads its HTML with jQuery-style selectors. Puppeteer drives a real Chrome for pages that build their content with JavaScript.

We tested all three on our own server on 12 October 2026, with Node.js 20.20.2, Cheerio 1.2.0 and Puppeteer 24.43.1. Every example below ran there, and its output is pasted under it. Two defaults surprised us. fetch's text() garbled both windows-1252 pages we served, and Node's fetch ignored the HTTPS_PROXY variable.

What you need

Node.js 18 or later, which ships fetch. Its documentation lists the change for version 18.0.0 as "No longer behind --experimental-fetch CLI flag". Then one package for HTML:

npm install cheerio

Add npm install puppeteer when you need a browser. When you install Puppeteer, "it automatically downloads a recent version of Chrome for Testing", its documentation says.

Scrape a page with fetch and Cheerio

This script reads the outline of one of our own posts:

import * as cheerio from "cheerio";

const res = await fetch("https://hproxy.com/blog/web-scraping-with-python", {
  headers: { "User-Agent": "hproxy-tutorial/1.0 (+https://hproxy.com/blog/javascript-web-scraping)" },
  signal: AbortSignal.timeout(30_000),
});
if (!res.ok) throw new Error(`HTTP ${res.status} for ${res.url}`);
const $ = cheerio.load(await res.text());

console.log($("h1").text().trim());
$("article h2").each((_, h2) => console.log("-", $(h2).text().trim()));

Web scraping with Python: requests, BeautifulSoup and proxies, step by step
- What you need
- Step 1: download a page, then read the bytes
- Step 2: find the data with CSS selectors
- Step 3: follow the pages, slowly
- Step 4: save the rows to a CSV file
- The whole scraper
- Step 5: send the scraper through a proxy
- When requests is not enough
- What this page could not check
- Sources
- Frequently asked questions

Save it as outline.mjs and run it with node outline.mjs. What each part does:

• The User-Agent names the scraper, so the site knows who is asking.
• AbortSignal.timeout() stops a request that hangs. Without it, fetch waits a long time, as the section on failures shows.
• res.ok matters because fetch does not throw for a 404 or a 500.
• cheerio.load() turns the HTML into a $ function that takes CSS selectors, the way jQuery does.

When the text comes out garbled

fetch's res.text() decodes every page as UTF-8, whatever the page says. The Fetch standard writes it into the method: text() returns the body after "UTF-8 decode". Most pages are UTF-8, but not all.

We served the same sentence four ways from a test server of our own, and read each page five ways:

On the windows-1252 pages, res.text() turned "Café crème, €3.20" into "Caf� cr�me, �3.20". axios gave the same result.

Cheerio's fromURL() reads the charset in the header, and both it and loadBuffer() read the meta tag, as a browser does. They also copy the browser's fallback for a page that declares nothing. The HTML standard's table of default encodings gives windows-1252 for "All other locales", and Chrome read our undeclared UTF-8 page that way too. loadBuffer() sees only the bytes, so it got the page with the charset in its header right only through that fallback. Only fromURL() with a UTF-8 default got all four right:

import * as cheerio from "cheerio";

const $ = await cheerio.fromURL("https://hproxy.com/blog/web-scraping-with-python", {
  encoding: { defaultEncoding: "utf-8" },
  requestOptions: {
    method: "GET",
    headers: { "User-Agent": "hproxy-tutorial/1.0 (+https://hproxy.com/blog/javascript-web-scraping)" },
    signal: AbortSignal.timeout(30_000),
  },
});
console.log($("title").text());

Web Scraping With Python: Step by Step, Tested | HProxy

requestOptions carries your headers and your timeout. Our test server received the User-Agent we set, and a 3 second timeout ended a request to a silent server after 3.0 seconds.

When fetch fails

fetch behaves differently from most HTTP clients when something goes wrong. Our test server showed four cases:

• A 404 is not an error to fetch. It returned the answer with res.ok set to false and res.status 404.
• A redirect is followed. With redirect: "manual", fetch returned the 302 itself and its Location header.
• A server that never answers kept fetch waiting. We stopped it after 15 seconds. undici, the library behind Node's fetch, waits up to 300 seconds for headers by default, by its documentation. With AbortSignal.timeout(5000) the request ended after 5.0 seconds with a TimeoutError.
• Nothing listening gives the vaguest message: TypeError: fetch failed. The reason sits in err.cause, in our test ECONNREFUSED.

A small helper that reports all of them:

async function getPage(url) {
  try {
    const res = await fetch(url, { signal: AbortSignal.timeout(10_000) });
    if (!res.ok) return `HTTP ${res.status}`;
    return `${(await res.text()).length} characters`;
  } catch (err) {
    return `${err.name}: ${err.message} (${err.cause?.code ?? err.cause?.message ?? "no cause"})`;
  }
}

console.log(await getPage("https://hproxy.com/robots.txt"));
console.log(await getPage("https://hproxy.com/this-page-does-not-exist"));
console.log(await getPage("http://127.0.0.1:59999/"));

910 characters
HTTP 404
TypeError: fetch failed (ECONNREFUSED)

Cheerio builds the same tree as a browser

HTML can be broken, or valid but written with closing tags left out, and parsers repair it in different ways. Cheerio parses with parse5 by default. Its documentation says parse5 "rigorously conforms to the HTML standard, so it produces the same tree a browser would".

We gave it the four bits of HTML that split BeautifulSoup's parsers in our BeautifulSoup tests. parse5 built the same body as Chrome for all four. htmlparser2, the parser Cheerio uses for XML, matched on those four as well. It differs on tables. A browser inserts a tbody into a table written without one. So #prices > tbody > tr found 2 rows with parse5 and in Chrome, and 0 with htmlparser2. Keep the default for HTML.

Pages that need a browser

Some pages arrive almost empty and fill themselves in with JavaScript. fetch and Cheerio only see what the server sent. We built a test page whose script adds five list items 150 ms after its HTML has loaded:

• fetch and Cheerio found 0 items.
• Puppeteer found 0 items right after goto() returned, and all 5 after waitForSelector(). Its documentation describes the method plainly: "Wait for the selector to appear in page".

The same outline as above, read through Chrome:

import puppeteer from "puppeteer";

const browser = await puppeteer.launch();
try {
  const page = await browser.newPage();
  await page.goto("https://hproxy.com/blog/web-scraping-with-python", { waitUntil: "domcontentloaded", timeout: 30_000 });
  await page.waitForSelector("article h2");
  const headings = await page.$$eval("article h2", (els) => els.map((el) => el.textContent.trim()));
  console.log(headings.length, "headings, the first:", headings[0]);
} finally {
  await browser.close();
}

11 headings, the first: What you need

A browser costs time and memory. We read 50 product pages of our own test server both ways, three times each:

Puppeteer took 18.87 seconds and fetch with Cheerio 0.27, both finding the same 2,000 prices. Chrome ran as 9 processes and used 785 MB of memory, against 126 MB for the Node.js process itself. So look at the HTML the server sends before you reach for a browser. If the data is in it, fetch and Cheerio are enough.

Scraping through a proxy

Node's fetch has no proxy option. In our test on Node.js 20, it also ignored the HTTPS_PROXY variable. The request went straight to the site, and our proxy's log stayed empty. Three setups did use the proxy:

EnvHttpProxyAgent reads the environment for fetch; by undici's documentation, "The uppercase variants HTTP_PROXY, HTTPS_PROXY, and NO_PROXY are also honored". The ProxyAgent version, with the proxy address in an environment variable:

import { fetch, ProxyAgent } from "undici";

const proxy = new ProxyAgent(process.env.PROXY_URL); // for example http://USER:PASS@premium.hproxy.com:10000
const res = await fetch("https://hproxy.com/robots.txt", {
  dispatcher: proxy,
  signal: AbortSignal.timeout(30_000),
});
console.log(res.status);

200

Our Node.js proxy guide tests these setups across Node versions and axios. Our node fetch guide covers rotation and SOCKS.

What this page could not check

• The test pages came from a server on the same machine, so no network delay was in the times.
• The times are the middle of three runs on one busy server. Puppeteer's 50 pages took 13.26 to 18.87 seconds across three runs.
• Puppeteer 24.43.1 drove the Chrome 143 already on our server, not the Chrome it would download itself.
• Node.js 20 only. Newer versions handle proxies differently, as our Node.js proxy guide shows.
• We will rerun these tests by 12 January 2027.

Sources

• WHATWG Fetch Standard, the text() method: fetch.spec.whatwg.org.
• WHATWG HTML Living Standard, determining the character encoding and the tbody rule: html.spec.whatwg.org.
• Node.js 20 documentation, the fetch global: nodejs.org.
• undici documentation, Client options and EnvHttpProxyAgent: github.com/nodejs/undici.
• Cheerio documentation, configuring Cheerio: cheerio.js.org.
• Puppeteer documentation, installation and Page.waitForSelector(): pptr.dev.
• MDN, AbortSignal.timeout(): developer.mozilla.org.
• Our own tests on our server, 12 October 2026: js_tests.mjs at 04:45 UTC and the code on this page at 04:42 UTC, against a test server of our own and hproxy.com.