Topic Data
Proxies for Crunchbase: exports, the API, free datasets and what the terms allow
Crunchbase's terms forbid crawling and scraping. Its paid plans export to CSV each month, the API is sold through sales, and two datasets are free.
Proxies for FBref: the rate limit, the terms, and football data the allowed way
Sports Reference blocks bots that send FBref more than ten requests a minute, and its terms bar competing databases and AI training. There is no API.
Proxies for Flashscore: the terms, robots.txt, Favorites and where its data comes from
Flashscore's terms allow personal use only and forbid scraping and aggregating without consent. Favorites send live alerts, and Opta supplies its football data.
Proxies for Google Play: the terms, robots.txt and prices by country
Google's terms tie automated access to robots.txt, which closes Play search and reviews. A store page shows other countries' prices when only gl changes.
Proxies for idealo: how its price comparison ranks offers, Price Alerts and its MCP server
How idealo ranks shop offers, how its Price Alerts and price history work, what its terms say about scraping, and the MCP server idealo runs for AI assistants.
Proxies for IndiaMART: the terms, the Leads API and why there is no Excel export
IndiaMART's terms ban systematic retrieval by robots or spiders without permission. Paid sellers can send their own leads to a CRM through its Leads API.
Proxies for Rightmove: the terms, property alerts and the open data on sold prices
Rightmove's terms forbid bots, crawlers and scrapers. Alerts watch new listings, Data Services offers data, and HM Land Registry publishes sold prices free.
Proxies for Substack: the terms, the RSS feed, exports and the MCP server
Substack's terms forbid crawling, scraping and spidering. Every publication has a public RSS feed, writers can export posts, and some can use an MCP server.
Proxies for Transfermarkt: the terms, robots.txt, the bot check and the RSS feeds
Transfermarkt's terms forbid bots, spiders and screen scraping, and an open robots.txt is no permission. What the site offers instead: free RSS news feeds.
Proxies for Yahoo Finance: the terms, the API terms and the paid download
Yahoo's terms forbid collecting data by robots, spiders or scrapers without its permission. Gold subscribers can download historical prices as a CSV file.
Proxies for Redfin: the scraping ban, MLS data and the free Data Center
Redfin's terms forbid automated crawling without written permission, and the MLS owns the listing data. What Redfin offers for free instead.
Proxies for Zillow: the terms on scraping, and Zillow's own data routes
Zillow's terms forbid automated queries and bypassing its captcha checks. What Zillow Research gives away free, and the APIs Zillow Group offers instead.
Proxies that don't die mid-job
Residential, ISP, datacenter and mobile, verified by the same engine that runs tens of millions of checks. They read as a real device and hold up under load. Pay as you go, and your balance never expires. $0.44/GB is the 2,000 GB+ rate; a single gigabyte is $0.50/GB, with no minimum order.
129M+ proxy checks run · 100+ countries · HTTP / HTTPS / SOCKS · re-checked every few minutes · no signup