About XPath

What is XPath?

The XML Path Language (XPath) is a set of syntax and semantics for referring to portions of XML documents. XPath is intended to be used by other specifications, such as XSL Transformations (XSLT), the XML Pointer Language (XPointer) and XML Query (XQuery). XPath expressions identify a node or a set of nodes in an XML/HTML document for working with.

XPath abbreviated directory

Nodes

In XPath, there are seven kinds of nodes: element, attribute, text, namespace, processing-instruction, comment, and document nodes. XML documents are treated as trees of nodes. The topmost element of the tree is called the root element. This set of nodes can contain zero or more nodes.

Look at the following XML document:

<?xml version="1.0" encoding="UTF-8"?>
<list>
    <item&nbsp;new=\'true\'>
        <title land="en">DVD Music</title>
        <author>K. A. Bred</author>
        <year>2012</year>
        <price&nbsp;currency="USD"&nbsp;>29.99</price>
    </item>
</list>

Example of nodes in the XML document above:

<list> (root element node)
K. Bred (element node)
lang="en",  new=\'true\' (attribute nodes)

Atomic values are nodes with no children or parent. Items are atomic values or nodes.

Parent. Each element and attribute has one parent. In the example above the item element is the parent of the title, author, year, and price.

Children. Element nodes may have zero, one or more children. In the example above, the title, author, year and price elements are all children of the item element.

Siblings are nodes that have the same parent. The the title, author, year, and price nodes are all siblings.

Ancestors are node s parent, parent s parent, etc. In the example above, the ancestors of the title element are the item element and the list element.

Descendants are node s children, children s children, etc. In the example, descendants of the list element are the item, title, author, year, and price nodes.

To learn how to apply these notations in real xpaths, refer to XPath in examples.

Attributes

XPath expressions can refer to attributes as well as elements in an XML/HTML document. When referring to an attribute, the @ character is used. For example, the following XPath expression identifies price elements whose currency attribute contains the value USD:

XPath evaluation relative to a context

XPath evaluates expressions relative to a context. Context is usually specified by the technologies that extend XPath, such as XSLT or XPointer. An XPath context includes a context node, context size, context position, and other context data. From a context standpoint, the context node is of most interest here since the scraping works for getting data from the node trees where access is for the context node-set.

Location paths

Location path is a specialization of an XPath expression. A location path identifies a set of XPath nodes relative to its context. XPath defines two syntaxes: the abbreviated syntax and the unabbreviated syntax. We consider only the abbreviated syntax because as the most widely used; the unabbreviated syntax is more complex.
The two types of location paths are relative and absolute.

A relative location path is a sequence of location steps separated by, for example:

• an axis (defines the tree-relationship between the selected nodes and the current node)
• a node-test (identifies a node within an axis)
• zero or more predicates (to further refine the selected node-set)

In the above example relative location path consists of three location steps: the first step list, selects a set of nodes relative to the context node; the second, item, selects a set of nodes in the subset identified by the first step, and so on. Note that there is no at the beginning of the path.

Absolute location path. It starts with a, optionally followed by a relative location path, with initial referring to the root node. An absolute location path is basically a relative location path evaluated in the context of the root node, for example:

In this example, we get all the item with price<20 that are children to list, which should be in a root node. With absolute location paths, the context node (current node set) isn t meaningful because path is always evaluated from root node.

Predicates

XPath expressions can include predicates. Predicates are used in location paths to filter the current set of nodes. A predicate contains a Boolean expression (or an expression that can be easily converted to Boolean). Each member of the current node-set is tested against the Boolean expression and kept if the expression is true. Otherwise, it is rejected. A predicate is enclosed in square brackets, []. Predicates are useful in refining the resulting node set. For example, 1st line XPath expression identifies the item only with price value more than 300, while 2nd line location path searches for price nodes with currency \’USD\’:

Predicates allow relational operators >, <, >=, <=, and != != as well as the use of Boolean operators. If < > are not allowed in some XSL expressions use &lt; or &gt; instead:

Note: &lt; is used instead of < .

Syntax

Expression Description

And some examples
@ is used to refer to attributes.

* (asterisk) is used to refer to all the elements that are children of the context node. Here 1st line xpath refers all the nodes children to item nodes; 2nd line xpath gets all the nodes:

[ ] (square-bracket) can also be used to refer to specific elements in an ordered sequence. The following example refers to the second item element:

// (double slash) is used to refer to all children of the context node. 1st line xpath chooses all item elements, while 2nd line xpath refers to all item elements that have a list parent:

. (dot) is used to refer to the context node itself (current node). The following xpath refers to all the item nodes that are children of the context node:

.. (double dot) is used to refer to parent of context node. This xpath refers to the item elements in the context of the author nodes (since item is a parent to author):

Operators

The | (pipe) operator allows you to get access to multiple node-sets. From our example, XML-set following xpath gets the node-set with all author and year elements:

Numeric operators provided by XPath are: + (addition), – (subtraction), * (multiplication), div (division), and mod (remainder from truncating division).

Strings in XPath are enclosed in quotation marks (\’ or “). When an XPath string is contained in an XML document and contains quotation marks, you have to use one of the two following options:

• Quote them using &apos; or &quot; respectively. For example:

  notes = \'New Lapytop&quots ;USB"
  MP3 player&quot\'

• Use single quotation marks (\’) if the expression is enclosed in double quotation marks (“), and vice-versa. For example

  select = "item[@private=\'false\'
  or price/@currency=\'USD\']"

Functions

XPath defines a set of functions called the core function library. Each function is identified by a function name, return type (must not be void) and the type of the arguments (zero or more, mandatory, or optional).

Functions are used inside predicates and expressions. XSLT extends this function set. The most used functions are divided into four groups:

Node-set functions provide information on a set of nodes (one or more nodes). Some node-set functions are:

• last() Returns a number called the context size, which is the number of nodes in a given context. (different from the last node)
• position() Returns a number called the context position, which is the position of the current node in the set (list) of nodes in a given context. For example, you can test whether you are dealing with the last node of a set with the expression position()=last().
• count(node-set) Returns the number of nodes in the argument node-set.
• id(object) Returns a node-set, the result of selecting elements by their unique id declared as type ID in a DTD (Document Type Definition).