The XPath function contains() matches part of a text or an attribute, which makes it a favourite for messy pages. It also fails quietly. Some uses match too much, some match nothing, and none of them raises an error.
On 12 October 2026 we ran 20 cases on a small test page of our own. We used three engines: lxml, parsel (the selector library inside Scrapy) and Chrome, and they agreed on every case.
The basics
from lxml import html
doc = html.document_fromstring(open("contains-page.html", encoding="utf-8").read())
print(len(doc.xpath("//a[contains(@title, 'Lamp')]")))
print(len(doc.xpath("//p[contains(., 'on sale')]")))
1
1
contains(a, b) is true when the string a contains b. The first argument can be an attribute, such as @title, or the element's own text, written as a dot.
Classes: contains(@class) matches too much
print([p.get("class") for p in doc.xpath("//p[contains(@class, 'price')]")])
print([p.get("class") for p in doc.xpath("//p[contains(concat(' ', normalize-space(@class), ' '), ' price ')]")])
['price', 'price_color', 'old-price', 'price sale']
['price', 'price sale']
The short form matched four elements, because contains() works on characters, not on words. Comparing with @class='price' went wrong the other way: it found only the element whose whole class is "price" and missed "price sale".
The longer form puts a space on each side of the class list and of the word, so only whole words match. It found the 2 right elements. In CSS, p.price does this for you; see our test of XPath against CSS selectors.
Text: use the dot, not text()
print(len(doc.xpath("//p[contains(text(), 'on sale')]")))
print(len(doc.xpath("//p[contains(., 'on sale')]")))
0
1
The test paragraph is <p>Price: <b>£10</b> on sale today</p>, and the first expression found nothing:
The paragraph holds three nodes, and text() selects the two text nodes. When contains() gets a set of nodes, XPath 1.0 keeps only one. The standard turns a set of nodes into "the string-value of the node in the node-set that is first in document order". So contains(text(), 'on sale') only looked at "Price: ". The dot stands for the whole text of the element, child tags included.
Case: contains() is case sensitive
LOWER = "translate(@title, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz')"
print(len(doc.xpath(f"//a[contains({LOWER}, 'lamp')]")))
2
contains(@title, 'Lamp') found only "Desk Lamp" and missed "desk lamp, black". XPath 2.0 has a lower-case() function, but lxml, parsel and Chrome speak XPath 1.0, and all three rejected it with an error. translate() swaps each capital letter for its small one, and then the search found both titles.
Spaces: normalize-space() and the non-breaking space
print(len(doc.xpath("//span[contains(normalize-space(.), 'Out of stock')]")))
print(len(doc.xpath("//span[contains(translate(., ' ', ' '), 'In stock')]")))
1
1
Pages often carry extra spaces, and normalize-space() folds each run into one, so "Out of stock" matched. It does not touch a non-breaking space, the of HTML. XPath counts as white space only what XML does: the space, the tab and the two line breaks. Our "In stock" with a non-breaking space matched only after translate() turned it into a plain space.
An empty value matches everything
sku = "" # a value that came back empty
print(len(doc.xpath("//*[contains(@data-sku, $sku)]", sku=sku)), len(doc.xpath("//*")))
17 17
Say the value comes from a variable and is empty. Then contains() is true for every element of the page, including the 16 without a data-sku attribute. The XPath rules explain why. A missing attribute becomes an empty string, and every string contains the empty string. Check the value before you search; passing it as an XPath variable, as here, does not change the result.
starts-with(), and ends-with() in XPath 1.0
from lxml import etree
try:
doc.xpath("//p[ends-with(@class, 'price')]")
except etree.XPathEvalError as e:
print(e)
print(len(doc.xpath("//p[substring(@class, string-length(@class) - string-length('price') + 1) = 'price']")))
Unregistered function
2
starts-with() is part of XPath 1.0 and found the 3 classes that begin with "price". ends-with() arrived with XPath 2.0, and all three engines refused it. The substring() form compares the end of the string instead and found the 2 classes that end in "price".
Any attribute, and two texts at once
print(len(doc.xpath("//*[@*[contains(., 'lamp')]]")))
print(len(doc.xpath("//p[contains(., 'Price') and contains(., 'on sale')]")))
1
1
@* means every attribute of the element, so the first line finds elements with "lamp" in any attribute. To require two texts, join two contains() with and.
The same in Selenium and in the browser
Selenium sends your XPath to the browser. The WebDriver standard says the browser runs it with its own evaluate(), the function behind our Chrome column. So in Selenium with Chrome, the results on this page hold. All 20 cases gave the same answers in lxml, parsel and Chrome:
These selectors usually run inside a scraper. Our Python scraping tutorial builds one. When a site limits your requests, see our guide to proxies for web scraping.
What this page could not check
- One test page we made, in three engines: lxml, parsel and Chrome 143.
- We did not run Firefox, Safari or an XPath 2.0 engine, and did not run Selenium itself.
- We will rerun these tests by 12 April 2027.
Sources
- W3C, XML Path Language (XPath) 1.0: w3.org.
- W3C, XML 1.0, white space: w3.org.
- W3C, XPath and XQuery Functions and Operators 3.1: w3.org.
- lxml, XPath and XSLT with lxml: lxml.de.
- W3C, WebDriver, the XPath locator strategy: w3.org.
- Our own tests on our server, 12 October 2026: contains_test.py at 00:26 UTC (lxml 6.1.3, parsel 1.12.1, Chrome 143) and the code on this page at 00:25 UTC.
