Python
Everything tagged Python.
62 articles
Topic Python
Tutorial
Best language for web scraping: Python, JavaScript, Go and the rest
We ran the same scraper in Python, JavaScript, Go and Java on our server: speed, memory, lines of code, broken HTML, timeouts and proxies, measured.
October 12, 2026 · 15 min read
Use case
Image scraping: how to download images from a website, and what is allowed
How to download a website's images with Python, tested: what img src misses, srcset and lazy loading, duplicates, robots.txt read the right way, and copyright.
October 12, 2026 · 7 min read
Use case
Job scraping: how to collect job postings and which sites allow it
How to collect job postings without breaking a site's rules: what 15 job sites' robots.txt files allow, what Indeed and LinkedIn's terms say, and the public job APIs.
October 12, 2026 · 10 min read
Tutorial
Open source web scrapers: what each project is good for
Which open-source web scraper fits which job: 18 projects with their licenses and releases, and 7 crawlers run with their defaults on a test site of our own.
October 12, 2026 · 12 min read
Tutorial
pandas read_html: scrape HTML tables into DataFrames
How to read HTML tables into pandas DataFrames with read_html, tested on pandas 3.0: picking a table, numbers, links, headers, and the errors you will meet.
October 12, 2026 · 6 min read
Tutorial
Parse HTML with regex: when it works and when to use a parser
Can you parse HTML with regex? We ran four regular expressions and two parsers on 13 small HTML inputs and checked every answer against Chrome's own parser.
October 12, 2026 · 7 min read
Tutorial
Parse JSON in Python: json.loads, files and API responses
How to parse JSON in Python, tested: strings, files and API answers, the 'Expecting value' error, NaN and big numbers in other languages, dates and orjson.
October 12, 2026 · 5 min read
Tutorial
Parse XML in Python: ElementTree, lxml and sitemaps
How to parse XML in Python, tested on our own sitemaps: the namespace trap that empties findall(), large files with iterparse, five parsers timed, and safety.
October 12, 2026 · 6 min read
Tutorial
PDF scraping: extract tables and text from PDFs with Python
PDF scraping in Python, tested on a 28-page IRS tax table: which library keeps rows in order, why the table finders failed, what fixed it, and scanned pages.
October 12, 2026 · 5 min read
Tutorial
Playwright wait for page load: load states, selectors and network idle
Which Playwright wait to use when a page loads its content with JavaScript: load states, networkidle, locator waits and responses, each timed on test pages of our own.
October 12, 2026 · 8 min read
Tutorial
Python HTTP clients compared: requests, httpx, aiohttp and urllib3
Six Python HTTP clients tested on one server: what each one sends, which follow redirects, which give up on a silent server, how fast they are, and what they install.
October 12, 2026 · 10 min read
Tutorial
Python requests headers: set, read and rotate them
What Python requests really sends as headers, tested on our own server: the five defaults, Sessions, tokens on redirects, rotation, br, and what a proxy sees.
October 12, 2026 · 5 min read
Tutorial
Python requests POST: form data, JSON and files
How to send form data, JSON and files with requests.post, tested against our own server: four calls that send something else, redirects, sessions and proxies.
October 12, 2026 · 5 min read
Tutorial
Python requests retry: retries, backoff and when to switch proxy
How to retry failed requests in Python, timed at our own server: no retries by default, the real backoff waits, Retry-After, POST, timeouts and proxy switching.
October 12, 2026 · 5 min read
Tutorial
Python web scraping libraries compared: what each one is for
The Python web scraping libraries that matter, tested on one job: requests and httpx, BeautifulSoup, lxml, parsel, selectolax, Scrapy, Playwright and Selenium.
October 12, 2026 · 11 min read
Tutorial
Selenium find element: locators, waits and the errors you will meet
How to find elements in Selenium with Python, tested on Chrome: the eight locators, find_element or find_elements, waits in seconds, and each error explained.
October 12, 2026 · 14 min read
Tutorial
Selenium headless mode: run Chrome headless, and what sites can detect
How to run Chrome headless in Selenium with Python, tested: which flag works now, the small default window, what a site can see, and how to fix each part.
October 12, 2026 · 10 min read
Tutorial
Web scraping with BeautifulSoup: parse HTML in Python, with examples
How to scrape with BeautifulSoup, tested: find and select, three parsers against Chrome, the tbody trap, r.content over r.text, and what makes it fast.
October 12, 2026 · 12 min read
Tutorial
Web scraping with Python: requests, BeautifulSoup and proxies, step by step
A Python scraper built step by step with requests and BeautifulSoup and run on 1,000 books: the encoding bug, timeouts, a CSV file and two proxy traps.
October 12, 2026 · 9 min read
Tutorial
Web scraping with Scrapy: spiders, pipelines and proxies
A Scrapy tutorial run end to end on a practice shop: a project, a spider, pipelines that drop bad rows, retries, speed settings, the HTTP cache and a proxy.
October 12, 2026 · 8 min read
Tutorial
Web scraping with Selenium in Python: setup, waits, proxies and blocks
How to scrape a JavaScript page with Selenium in Python, tested on a practice shop: waits, reading in one call, every page to CSV, and a proxy with a login.
October 12, 2026 · 10 min read
Tutorial
XPath contains(): match text and attributes, with examples
XPath contains() tested in lxml, parsel and Chrome: why contains(text()) misses split text, why class matching overshoots, not(), case, spaces and empty values.
October 12, 2026 · 6 min read
Tutorial
cURL to Python: convert a curl command to requests and httpx
How to convert a curl command to Python requests or httpx, option by option, and the defaults that change on the way. Every command run on October 11, 2026.
October 11, 2026 · 10 min read
Tutorial
XPath vs CSS selector: which to use for scraping
XPath vs CSS selectors, tested on one page in lxml, parsel, BeautifulSoup and Chrome: what each can find, the class trap that drops matches, and which is faster.
October 11, 2026 · 6 min read
Showing 1 to 24 of 62
Proxies that don't die mid-job
Residential, ISP, datacenter and mobile, verified by the same engine that runs tens of millions of checks. They read as a real device and hold up under load. Pay as you go, and your balance never expires. $0.44/GB is the 2,000 GB+ rate; a single gigabyte is $0.50/GB, with no minimum order.
129M+ proxy checks run · 100+ countries · HTTP / HTTPS / SOCKS · re-checked every few minutes · no signup
HProxy.
Honest guides and comparisons on proxies, scraping and staying unblocked, from the team that runs the network.