Use case

Proxies for Indeed: the Site Rules, robots.txt and open Hiring Lab data

Indeed answered our server with a security check. Its Site Rules bar scrapers without written permission, its APIs serve employers, and Hiring Lab data is open.

HProxy Team··Updated September 28, 2026·4 min read
HProxy.Use case

Free proxies won't hold up here.

Shared datacenter IPs get flagged and dropped fast. When it has to hold, gaming, streaming, accounts, you need mobile and residential IPs that read as a real device, from $0.44/GB, pay as you go.

Proxies for Indeed→

People who look for Indeed proxies usually want job data: postings, salaries or hiring trends. This page answers from the Indeed Terms of Service, its robots.txt file and the open data of Indeed Hiring Lab. It also reports what Indeed showed our server. The short answer: Indeed bars bots and scrapers without its written permission. Its APIs serve employers, and the Hiring Lab data is open to anyone who credits it.

What Indeed showed our server

We sent plain requests from our server in St. Louis on 28 September 2026, with a browser user agent and no JavaScript.

requestanswer
the Indeed front pageHTTP 403, a page titled Security Check
the Terms of ServiceHTTP 200, the full terms
robots.txtHTTP 200, the file
the developer documentationHTTP 403, the same check

So a plain request from a data center never reached the job search. We did not try to get past the check. The terms say that Indeed "reserves the right to use technical measures to detect, investigate, or prevent unauthorized automated access or activity on the Site".

What the Indeed Site Rules say

The Terms of Service were last updated on 17 July 2026. Their Site Rules are a list of things you may not do, directly or indirectly. Three of them cover proxies and scrapers.

  • "Access the Site through any means other than the public interfaces provided by Indeed".
  • "Bypass any security measures or Site limitations".
  • "Use any automated system (bots, scrapers, spiders, AI or Agentic AI) to access, data-mine, or submit content to the Site, in bulk or otherwise, without Indeed’s express written permission".

The last rule goes on: "You may not crawl, scrape, extract data from, reproduce, duplicate, copy, sell, exploit, trade or resell any part of the Site". Indeed does grant one permission: to crawl "solely as outlined in our robots.txt file".

What robots.txt allows

The Indeed robots.txt file sets those crawl rules. It holds 480 Disallow lines and 37 Allow lines, with a default group for all unnamed bots. One group, labelled "Full disallow for aggressive scrapers, legacy crawlers, and unwanted utility bots", disallows the whole site. It names Scrapy, a common scraping framework, among others.

The official routes: APIs for employers, open data from Hiring Lab

The Indeed APIs sit under their own API terms, for employers and partners. Indeed "monitors and limits API usage", and "Access to the Sponsored Jobs API is limited to users and End Users who actively spend to sponsor a Job Ad through a monthly campaign". The developer documentation answered our server with the security check, so we could not read it.

For job-market data, Indeed Hiring Lab, the research arm of Indeed, publishes the data behind its Job Postings Index on GitHub. The figures are daily and "refreshed each week". They measure the change in job postings since 1 February 2020.

filewhat it holds
aggregate job postingsone index per country
job postings by sectorthe index per occupational sector
US metro areasmetro areas of at least 500,000 people
US statesthe states and the District of Columbia
regional filesCanada, the UK, Australia, Germany, France and Ireland

The license is Creative Commons Attribution 4.0. In the words of Hiring Lab, the data are "freely available for public use", as long as Indeed Hiring Lab is cited as the source.

If you use proxies for other work

Nothing on this page routes Indeed traffic through a proxy, because the Site Rules bar scrapers without written permission. For other job and research work, see our pages on proxies for Glassdoor and proxies for market research. For scraping on sites that allow it, see our guide to proxies for web scraping and whether free proxies for web scraping hold up. For the basics, see what a residential proxy is, whether free proxies are safe and how to check if a proxy is working. Our free proxy list and proxy checker cost nothing. Our residential, ISP and mobile proxies are on the pricing page, and residential traffic starts at $0.44/GB.

The limits of this page

We made plain requests without JavaScript, from one server in the United States. We did not log in, search for jobs, or ask Indeed for permission. The developer documentation and the Hiring Lab website answered our server with the security check, so the API facts come from the terms and the data facts from the Hiring Lab repository. This page is due for a check by 28 December 2026.

Sources

We read every source below on 28 September 2026.

More pages in this series: proxies for Glassdoor, proxies for market research and proxies for Kayak.

Frequently asked questions

Can I scrape Indeed?
Not without written permission. The Indeed Site Rules bar any automated system, from bots and scrapers to AI agents, from accessing or data-mining the site without express written permission, and they bar bypassing security measures or site limits.
Does Indeed have an API for job data?
Its APIs serve employers and partners under the Indeed API terms, such as the Sponsored Jobs API, which is limited to employers who pay to sponsor a job ad each month. For job-market data, Indeed Hiring Lab publishes an open job postings index.
What is the Indeed Hiring Lab data?
The data behind the Hiring Lab Job Postings Index: daily figures, refreshed each week, by country, occupational sector and some regions. It is free for public use under the Creative Commons Attribution 4.0 license, as long as Indeed Hiring Lab is cited.
Why does Indeed show a security check?
Indeed reserves the right to use technical measures to detect, investigate or prevent unauthorized automated access. Our plain request from a server in the United States got HTTP 403 and a page titled Security Check.

Proxies that don't die mid-job

Residential, ISP, datacenter and mobile, verified by the same engine that runs tens of millions of checks. They read as a real device and hold up under load. Pay as you go, and your balance never expires. $0.44/GB is the 2,000 GB+ rate; a single gigabyte is $0.50/GB, with no minimum order.

129M+ proxy checks run · 100+ countries · HTTP / HTTPS / SOCKS · re-checked every few minutes · no signup

HProxy.

Honest guides and comparisons on proxies, scraping and staying unblocked, from the team that runs the network.

RSS feed