Travel fare aggregation pulls flight prices from airlines, travel agencies and metasearch sites into one comparable feed. The sites' own rules come first. Of 12 fare sites whose robots.txt we read on October 25, 2026, 7 disallow their flight search page for all crawlers. KAYAK's terms forbid scraping without its written permission. Licensed routes exist: Duffel charges $3.00 per order with 1,500 searches allowed per order, and Skyscanner admits partners by application. Fares can change by the market of a search, which is where country-targeted proxies come in. On the sellers' pages we read on October 26, 2026, country-targeted residential traffic cost from $0.60 per GB for 10 GB. Our Lite pool costs $0.50/GB from one GB.
The sections below cover the rules, the licensed routes, why the market changes a fare, and which proxies fit. Our guide on how to scrape flight prices covers the collection mechanics in depth.
Do fare sites allow automated collection?
Mostly not on their search pages. We read the robots.txt of 12 fare sites on October 25, 2026. For each site, we checked one flight search path against its rules for all crawlers:
- Disallowed
- No matching rule
The closed ones were Google Flights, Kayak, Momondo, Skyscanner, Expedia, Air France and Ryanair. Priceline, United, Delta, American and Southwest had no rule matching their search path. A site's terms can say more than its robots file.
KAYAK's terms were updated on August 17, 2026. Users agree not to "scrape, harvest, deep-link to, train AI on, or otherwise use automated tools on our Site without our written permission". Courts have looked at fare scrapers too. The Ninth Circuit's 2022 hiQ opinion cites Southwest Airlines v. FareChase. That 2004 ruling held a scraper "to glean flight information" unauthorized, because it interfered with Southwest's use of its site.
A proxy changes where a request comes from. It grants no permission, and the risk of breaking a site's rules stays with whoever sends the requests.
Which licensed routes give fare data?
Several, each with its own terms and price:
| Route | What it gives | The terms |
|---|---|---|
| Skyscanner APIs | live fares by market | partners, by application |
| Duffel | search and booking | $3.00 per order |
| Travelpayouts Data API | cached Aviasales fares | a token, 7-day cache |
| Google QPX Express | fare search | closed in April 2018 |
Duffel's price rule rewards booking. It allows 1,500 searches per confirmed order each month, then charges $0.005 per extra search. Travelpayouts serves fares found by Aviasales users, stored for 7 days, with each search tied to a market.
Why does the market of a search change the fare?
Because a fare can be filed for some markets and not others. Travelport's Category 15 rules restrict the sale of a fare by "countries/currencies of sale". They also restrict it by "locales where the fare may or may not be sold". Skyscanner's own fare search takes the market as a required field, "where search is coming from", next to the currency.
So an aggregator that compares markets needs each check to come from the market in question. That means a licensed route that accepts a market, or an exit in that country on a site that allows the visit. Our study of whether proxies or VPNs get cheaper flights shows what published tests found.
What breaks in fare aggregation?
- Cached prices. Thomas Vissers and colleagues found that "travel companies do not display real time price information". Prices are cached, so a listed fare can differ from the bookable one.
- Vanishing fares. The same study saw flights "suddenly become unavailable, only to reappear in the next measurement", about a minute apart.
- Sessions mid-flow. A multi-step booking flow needs one address throughout. Our explainer on rotating and static residential proxies covers sticky sessions.
- The rules change. Robots files and terms move without notice, so read them before each project.
What are the best residential proxies for fare checks by market?
The best fit targets the countries whose fares you compare and holds a session for a flow. We read each seller's own pricing page on October 26, 2026:
| Seller | Price per GB, Oct 26 | Why it fits fare checks |
|---|---|---|
| HProxy Lite | $0.50/GB from 1 GB, $0.44/GB from 2,000 GB+ | country, never expires |
| Rainproxy | $0.60 at 10 GB, $0.40 at 100 GB | traffic never expires |
| Evomi | $0.99 from 15 GB | city, sticky sessions |
| DataImpulse | $1.00 from 5 GB | free country targeting |
| MarsProxies | $3.99 at 10 GB | city, sticky sessions |
| HProxy Premium | $1.49/GB from 1 GB | city, 24-hour sessions |
Decodo targets down to ZIP code on monthly plans from $3.75 per GB at 3 GB. Bright Data opens its residential network only to verified companies, at $8 per GB pay as you go. Our best residential proxies comparison sets ten sellers side by side. Hotel rates follow similar rules, covered in how to scrape hotel prices.
What does HProxy offer for travel data?
Two residential pools billed by the GB, with a balance that never expires. Residential Lite targets by country, from $0.50/GB for one GB down to $0.44/GB at 2,000 GB+. Premium adds state, city and ASN targeting with sticky sessions up to 24 hours, from $1.49/GB. Our pricing page lists every product, and unlimited residential proxies explains why residential traffic is metered.
What this page could not check
- No test of any network. We measured no speed or success rate at any seller, ours included, and ran no fare collection.
- One path per site. The robots reading checks one flight search path at each of 12 sites, on one day.
- Sellers as printed. The seller table repeats what each pricing page printed on October 26, 2026. We bought from none of them.
- No legal advice. Terms and court records are quoted for what they say.
- Check by January 26, 2027. Robots files, terms and prices change, and every figure here needs a fresh read by then.
Sources
- HProxy, robots.txt files of 12 fare sites, read October 25, 2026.
- KAYAK, Terms of Use, updated August 17, 2026, read October 25, 2026.
- US Court of Appeals for the Ninth Circuit, hiQ Labs v. LinkedIn, No. 17-16783, opinion of April 18, 2022, citing Southwest Airlines v. FareChase (2004).
- Skyscanner, developer documentation: Authentication and Flights Live Prices, read October 25, 2026.
- Duffel, pricing page, read October 25, 2026.
- Travelpayouts, Help Center, Aviasales Data API, read October 25, 2026.
- Google Developers, QPX Express FAQ, archived capture of November 1, 2017.
- Travelport, Fares and Pricing help, Category 15 Sales Restrictions, read October 25, 2026.
- Thomas Vissers, Nick Nikiforakis, Nataliia Bielova and Wouter Joosen, "Crying Wolf? On the Price Discrimination of Online Airline Tickets", HotPETs, July 2014.
- Rainproxy, Evomi, DataImpulse, MarsProxies, Decodo and Bright Data, residential pricing pages, read October 26, 2026.
- HProxy, residential product pages and price list, read October 26, 2026.
