Glossary · Detection & reputation
Cloudflare
A CDN and security layer that fronts a large share of the web and includes some of the most common anti-bot challenges scrapers meet.
Cloudflare sits in front of a very large fraction of websites as a CDN and security proxy, which means its bot-defence is one of the most common obstacles automation runs into. When a request looks automated, Cloudflare can interpose a challenge, a JavaScript check, a managed challenge, sometimes a CAPTCHA, before the real site is reached.
Because Cloudflare is a reverse proxy for the site, what you are dealing with is its defence, not the origin's. That defence weighs the same signals every serious anti-bot system does: IP reputation, TLS fingerprint, browser behaviour. A datacenter address draws a challenge far more readily than a clean residential one.
The practical upshot is unchanged: a trusted address, a genuine browser fingerprint and unhurried behaviour get through where any one of them alone does not, and for the hardest configurations a purpose-built scraper endpoint is often the pragmatic route.
Back to the full glossary.