Proxies for Price Monitoring: Which Type, How Much Traffic, What It Costs

A price monitor needs an exit in the market whose prices it reads. Its bill follows from how much of each page it downloads. Datacenter proxies are enough for price APIs and open shops. Rotating residential proxies are the usual pick where a shop prices by location or turns hosting addresses away. One month of daily checks on 1,000 product pages moves about 1.1 GB if the monitor reads only the HTML. A browser that loads every page in full moves about 92 GB, by HTTP Archive's page weights of October 2026. At the median residential price of $3.50 per GB, that is about $4 against about $321 a month.

Every number below names its source. Our web scraping guide covers the basics this page builds on.

What proxies do you need for price monitoring?

The target decides, and many monitors use two kinds. An official price API needs no proxy at all. A datacenter proxy is the cheap choice for shops that neither price by place nor refuse hosting addresses. Rotating residential proxies cover the shops that do.

Mobile proxies cost the most, a median $4.00 per GB on the smallest plan, and a monitor rarely needs one. In a 2014 study, Home Depot steered phone browsers toward more expensive products. That came from the browser's User-Agent alone, because every request left from the same block of addresses in Boston. To read what a phone user sees, send a phone browser's headers.

Do prices really change by country or city?

At some shops, for some products. In 2012, researchers changed nothing but the visitor's IP address, through proxies in six places, and compared 600 products at 200 sites. For most products they found no significant difference. Three sites priced by place: Amazon for Kindle e-books, Steam for games and Staples for office supplies.

Where the gaps existed, they were large. At Amazon, most of the Kindle e-books in the sample differed by at least 21%, and the widest gap reached 166%. At Steam, 20% of the games cost something else in Spain and Germany. Within Massachusetts alone, Staples showed 15 of 29 products at up to 11% above the lowest price, depending on the ZIP code.

A 2013 follow-up asked 340 shoppers in 18 countries to flag prices, then crawled 21 shops daily for a week. Most of the shops that varied did so by 10% to 30%. The authors could not explain the gaps by currency, shipping or taxes.

Location is one input among several. A 2014 study of 16 large shops and travel sites found personalization on nine. Members saw lower hotel prices at Cheaptickets and Orbitz. To rule out location, the authors sent every request from one block of addresses. Their reason: "e-commerce sites may personalize results based on a user's geolocation".

For a monitor, that gives the design rule: one fixed exit location per price series. If the exit city changes during a series, a change in price may come from the move.

Shelf and web prices differ too. At 56 large retailers in 10 countries, about 72% of online prices matched the store. The match was weakest at drugstores and office-supply chains, which price each store differently. Such chains show a local price once a store or ZIP code is set. In the 2012 study, Staples asked visitors from outside the US for a ZIP code. A monitor of those chains sets the same store every time.

Residential or datacenter proxies for price monitoring: which works?

Datacenter proxies work wherever a site does not judge the network. They cost a median $0.60 per GB on the smallest plan, against $3.50 for residential, across the sellers in our price records. Shops that refuse hosting networks, or price by place, need a residential exit in the right market. Our page on datacenter vs residential proxies shows how sites tell the two apart.

The address is one signal of several. In our test of 52 US retailers on August 24, 2026, 17 served a normal home page to a plain script. Most of those still held their product data back behind a sensor cookie or a behavior score. Nine answered our home and datacenter requests differently. A controlled re-test on October 22 then changed one thing at a time. The address changed the answer at none of 14 sites, and a change of the client's TLS stack alone changed two.

So start with a real browser on an address in the shop's own market. Keep the cheaper datacenter route for the targets that accept it. Our retail bot-defense study lists which defense each retailer runs. For the mechanics of pools, gateways and sticky sessions, see rotating vs static residential proxies.

How much traffic does price monitoring use?

As much as the pages you load. HTTP Archive records what web pages weigh in a regular crawl. In its crawl of October 1, 2026, the median desktop page came to 2,987 KB with every image, script and font. Its HTML document alone came to 35 KB. A monitor that reads the HTML, or the price API the page calls, moves a small share of what a browser moves.

HTTP Archive weighs home pages, so product pages can come out heavier or lighter. Retries and block pages add bytes of their own, which these figures leave out. Measure a sample of your real targets before you buy. Our headless browser bandwidth study shows how to count the bytes of each load in Playwright.

How much do proxies for price monitoring cost per month?

Multiply the traffic by the price per GB. For the job above, on our Residential Lite plan at its one-GB rate:

These are ceilings, because Lite steps down from 100 GB, to $0.44/GB at 2,000 GB+. City or ISP targeting needs Residential Premium, at $1.49 per GB for one GB. At the market's median of $3.50 per GB, the same four jobs cost about $4, $321, $648 and $1,346 a month.

The type of proxy moves the bill as much as the page does:

A monitor that runs through a few fixed addresses can skip per-GB billing. A static ISP address cost a median $3.00 a month across 24 sellers, and a datacenter address $1.77 across 18. We price static ISP at $2.70/IP and datacenter IPv4 at $1.58/IP for a month, with traffic left unmetered. Our page on unlimited residential proxies explains why residential traffic is metered.

What does HProxy's residential plan include for price monitoring?

Two residential pools, both billed by the GB:

A card payment starts at $10 and a crypto payment at $3, and both fund one balance. The payment options page lists every way to pay. The residential proxies page and the residential pricing page carry the full ladders, and pricing lists every product.

What are the best residential proxies for price monitoring in 2026?

The best fit targets the countries or cities you check and keeps unused traffic for the next run. We read each seller's own pricing page on October 26, 2026:

A monitor with a steady load can take a monthly plan instead. Evomi's costs $49 for 100 GB, or $0.49 per GB. City checks cost more at the cheapest sellers. Rainproxy prices city and ASN targeting on a separate pool from $2.98 per GB. Our best residential proxies comparison sets ten sellers side by side.

How do you set up a proxy for a price monitor?

1. Pick the markets. Set one exit location per market in the login, and keep it for the whole series.
2. Read the rules. Read robots.txt and the terms, and look for an official price API first.
3. Weigh a sample. Load 20 real pages the way the monitor will, and count the bytes.
4. Set the schedule. Check fast-moving items often and the rest less often.
5. Log the failures. Count 429s, block pages and empty prices on every run.

Each line from our dashboard carries its own login. On the Premium gateway, port 9000 gives a new address on each request, and each port from 10000 upward holds one sticky session:

# One price check through the Premium gateway. The login is copied whole
# from the dashboard, where the country, state or city is chosen.
import time
import requests

PROXY = "http://USER:PASS@premium.hproxy.com:10000"  # sticky: one address per flow

def fetch_price_page(url):
    r = requests.get(url, proxies={"http": PROXY, "https": PROXY}, timeout=30)
    if r.status_code == 429:
        wait = r.headers.get("Retry-After", "")  # seconds, or an HTTP date
        time.sleep(int(wait) if wait.isdigit() else 300)
        return None  # skip this item; the next run tries it again
    r.raise_for_status()
    return r.text

Test the line on the free proxy checker before the first run. Our guides to Playwright and Puppeteer show the same login in a browser.

How often should you check competitor prices?

As often as the prices move, within what the site allows. Daily scraped prices from one supermarket site per country, collected from 2007 to 2010, show how long an online price usually lasted:

These are implied averages from Alberto Cavallo's study of 80,000 products. In the US data, about 2.4% of products changed price on a given day. A daily pass over a whole catalog therefore reads mostly unchanged prices. Check the items that move often more often, and read the rest less often.

Large programs work this way. By 2010, the Billion Prices Project at MIT collected 5 million prices a day from more than 300 retailers in 50 countries. The prices that change by market feed straight into market research.

Is price scraping legal?

It depends on the country, the site's terms and what you do with the data, and this section is no legal advice. The leading US case is hiQ Labs v. LinkedIn. In April 2022, the Ninth Circuit found it likely that reading public data "will not constitute access without authorization under the CFAA". The CFAA is the federal anti-hacking law. The same opinion says trespass to chattels claims "may still be available", and that breach of contract and other claims "may also lie".

The robots.txt file is a site's published request to crawlers. Its standard, RFC 9309, says those rules "are not a form of access authorization". A monitor should read the file and honor it. It should also fetch the file again at least once a day, because the standard caps a cached copy at 24 hours.

Where a site offers an official channel, use it. Amazon's Product Pricing API supports repricers that "continuously monitor price points, competitor prices, and other relevant factors", and it is open to sellers only. Best Buy's Products API covers more than one million products, and Best Buy says most of that data, prices included, is updated "near real-time". Our proxies for Amazon guide covers Amazon's routes, and the Amazon and Best Buy price tracker guides show the official ones at work.

What should a price monitor do when a site answers 429?

Wait, then lower the rate for that site. A 429 means the client "has sent too many requests in a given amount of time", in the words of RFC 6585. It may carry a Retry-After header with the wait, given in seconds or as a date, under RFC 9110. The standard leaves open how a server counts requests, so a site may count them per account, per cookie or across many servers.

Log every 429 with the time and the site. A rising count means the schedule is too tight for that site. Our guide to the 429 status covers who counts and how backoff works. How a rate limit becomes a ban shows the request habits sites tolerate.

What breaks in a price monitor, and how do you catch it?

Four failures cause most bad data:

• The page changes its layout. The Billion Prices Project calls watching "the performance of the robots over time" the main work of scraping. Compare each run's count of prices found with the last run.
• The price needs a browser. In our 52-retailer test, most of the 17 sites that served a plain script a home page still gated their product data. A browser earns the cookies those sites check, at the traffic cost above, and the price API a page calls is lighter when it exists.
• The wrong market answers. An exit in the wrong place or a missing ZIP code returns a working page with the wrong price. Store the currency, the store and the exit location with every price.
• A page without a price. A page can load in full and still hold no price. Check that the price field holds a number before you store it.

What this page could not check

• No test of our own. We ran no price-monitoring job and measured no block rate, success rate or speed at any retailer or seller, ours included.
• Old studies. The location studies date from 2012 to 2014 and covered a few hundred sites. Test whether your own targets price by place.
• Home pages. HTTP Archive weighs home pages once a month, and product pages, retries and block pages change the totals.
• One shop per country. The price durations come from one supermarket site per country, 2007 to 2010.
• Mixed per-IP plans. The per-address medians mix shared and dedicated plans, and the API pages may change their terms.
• No legal advice. The court opinion is quoted for what it says.
• Sellers as printed. The seller table repeats what each pricing page printed on October 26, 2026. We bought from none of them.
• Check by January 25, 2027. Prices, page weights and API terms move often, and every figure here needs a fresh read by then.

Sources

• Jakub Mikians, Laszlo Gyarmati, Vijay Erramilli and Nikolaos Laoutaris, "Detecting price and search discrimination on the Internet", ACM HotNets, October 2012.
• Jakub Mikians, Laszlo Gyarmati, Vijay Erramilli and Nikolaos Laoutaris, "Crowd-assisted Search for Price Discrimination in E-Commerce: First results", ACM CoNEXT, 2013.
• Aniko Hannak, Gary Soeller, David Lazer, Alan Mislove and Christo Wilson, "Measuring Price Discrimination and Steering on E-commerce Web Sites", ACM Internet Measurement Conference, November 2014.
• Alberto Cavallo, "Scraped Data and Sticky Prices", NBER Working Paper 21490, August 2015.
• Alberto Cavallo and Roberto Rigobon, "The Billion Prices Project: Using Online Prices for Measurement and Research", Journal of Economic Perspectives, Spring 2016.
• HTTP Archive, page weight reports, crawl of October 1, 2026, read October 25, 2026.
• IETF, RFC 6585, "Additional HTTP Status Codes", April 2012, section 4.
• IETF, RFC 9110, "HTTP Semantics", June 2022, section 10.2.3.
• IETF, RFC 9309, "Robots Exclusion Protocol", September 2022.
• US Court of Appeals for the Ninth Circuit, hiQ Labs v. LinkedIn, No. 17-16783, opinion filed April 18, 2022.
• Amazon, Selling Partner API documentation, "Product Pricing API", read October 25, 2026.
• Best Buy, developer portal, "Our APIs", read October 25, 2026.
• HProxy, "We Tested 52 US Retailers' Bot Defenses", August 24, 2026, corrected October 22, 2026.
• HProxy price records: the price lists of 66 residential, 33 mobile, 17 datacenter and 11 rotating ISP sellers, captured July 2 to October 22, 2026.
• HProxy residential, payment and gateway pages, read October 25, 2026.
• Evomi, DataImpulse, Rainproxy, MarsProxies and Decodo, residential pricing pages, read October 26, 2026.