Abstract

XML structures data into a rather small number of different constructs, most notably elements and attributes. The XML Path Language (XPath) defines a way how to select parts of XML documents, so that they can be used for further processing. XPath's primary use in in XSL Transformations (XSLT), but other XML technologies use it as well, e.g. XSDL. XPath is a very compact language with a syntax that resembles path expressions well-known from file systems. These path expressions, however, are generalized and therefore much more powerful than the rather simple path expressions in file systems. Because of its use in different XML technologies, XPath is one of the most important XML core technologies.

Selecting Parts of XML Documents

• XML is a syntax for trees
  • it defines a way for how trees can be exchanged
• XML technologies should provide support for working with trees
  • when receiving trees, access to the tree should be easy (DOM)
  • validating trees should be easy (XSDL)
  • mapping trees should be easy (XSLT)
  • querying tree collections should be easy (XQuery)
  • XPath is what regular expressions for text-based information

Making Selection Reusable

• Different XML technologies need selection
  • XSLT needs it for selecting parts and manipulating them
  • XSDL needs it for applying identity constraints
  • DOM needs it for extracting parts from an XML tree
  • XQuery needs it for writing XML-oriented queries
• XPath was created to be reusable
  • XML experts should only learn one selection language
  • this knowledge can be reused when learning new technologies
  • implementations can reuse code libraries

How XPath Evolved

• XSL was designed as the new XML stylesheet language
  1. XSL Transformations (XSLT) transform the input document
  2. XSL Formatting Objects (XSL-FO) is what they will transform it to
• XSLT was designed to work on arbitrary XML input documents
  • started as a part of XSL (WD-xsl-19981216 → WD-xslt-19990421)
  • for selecting parts of the transformation input, a selection mechanism had to be provided
• XPath was turned into a standalone specification
  • started as a part of XSLT (WD-xslt-19990421 → WD-xslt-19990709)
  • reused in a number of other W3C specifications (XSDL, DOM)
  • Complete overhaul for XSLT 2.0 and XQuery
    • XPath 2.0 as the core language
    • a much larger set of functions and operators
    • the underlying data model which describes the foundation

Starting from the Infoset

• XPath operates on an abstract data model
  • a tree derived from the XML Information Set (XML Infoset)
  • a simplification (another one!) of the underlying XML
• The Infoset is turned into an XPath node tree
  • 11 infoset item types → 7 XPath node tree node types
  • character items are merged into text nodes
  • namespace declarations are no longer visible as attributes

What is Not in the XPath Tree

• The same things which are not in the Infoset
  • the order of attributes in a start tag
  • the types of quotes around attribute values
  • character references and entities (ü/ü → ü)
• And some more …
  • namespace declarations are no longer visible as attributes
  • notations and unexpanded entity references

Tree In / Selection Out

• XPath evaluates an expression based on a tree
• Where the tree comes from is out of XPath's scope
• The result of the evaluation is a selection
  • //img[not(@alt)] → select all images which have no alt attribute
  • count(//img) → return the number of images
  • /descendant::img[3]/@src → return the third image's src URI
  • starts-with(/html/@lang, 'en') → test whether the document's language is english
• Syntax errors may occur

Location Path Structure

• Each location path consists of Location Steps
  • location steps are separated by /, like path names in file systems
• Similarities between XPath location paths and file systems
  1. nodes in the XPath tree have different types
  2. the type and number of nodes selected by one step
  3. the direction in which each step moves
  4. additional filters for selecting specific nodes
• Differences between XPath location paths and file systems
  1. XPaths may return other data types than nodes
  2. XPath provides a built-in function library

File System vs. XPath Paths

Tests for Nodes

• Name tests
  • testing for a particular name (elements/attributes): /html/head/title
  • wildcards (testing for any name): /html/head/*
• Node type tests
  • text nodes: text()
  • comment nodes: comment()
  • any nodes: node()
• Processing instruction tests
  • any PI: processing-instruction()
  • specific PI: processing-instruction("xml-stylesheet")

Where Do You Want to Go Today?

• File system paths are one direction only
  • always one level down in the file system hierarchy
  • . and .. are clever directory shortcuts
  • other directions supported by tools (e.g., find)
• XPath allows steps is different directions
  • the default direction is child
  • other directions are explicitly specified: descendant::a

Axis Peculiarities

• Attributes and Namespaces are not the children of elements, but …
• … elements are their attributes' parent!
  • very counter-intuitive
  • very convenient
• Attributes and Namespaces are always leaves in the node tree
• Attribute nodes have the attribute value as their value
• Namespace nodes have the namespace name (i.e., a URI) as their value
• Namespace nodes exist because of namespace declarations
  • in the XPath node tree, only the namespace nodes are visible
  • the namespace declaration attributes (xmlns) are invisible
  • one namespace declaration potentially creates many namespace nodes

Axes

Putting it all Together

• XPath location paths use a simple syntax
  • sequence of location steps, separated by /
• Each location step uses a simple structure (preceding::p[@class="warning"])
  1. an axis followed by :: (no axis uses the default axis child)
  2. a node test
  3. 0-n Predicates enclosed in []
• Location paths can be abbreviated
  • child:: can be omitted (default axis)
  • attribute:: can be written as @
  • . is an abbreviation for self::node()
  • .. is an abbreviation for parent::node()
  • // is an abbreviation for /descendant-or-self::node()/

Location Step Filters

• Predicates are filters for each location step
  • there can be any number of filters (0-n)
  • each filter is applied to each selected node individually
• Each predicate is an XPath and evaluated as a boolean
  • the context of this evaluation is the node for which the filter is evaluated
  • if the result is a number, it is compared with the position() function (/descendant::a[5])
• Predicates always reduce the set of selected nodes
  • as corner cases, the set of selected nodes does not change or is empty
  • predicates are used in the majority of non-trivial XPath location paths

Location Path Processing

• Location paths are processed in a very simple way
  1. start with a given context
  2. for each location step, repeat the following steps:
  3. based on the context and the axis, select the nodes on this axis
  4. reduce this selection to the nodes identified by the node test
  5. sequentially apply all filters to each of these nodes
  6. take the remaining node set as the context for the next location step

Beyond Location Paths

• XPath is a full expression language
  • any evaluated expression in XSLT is an XPath
  • XPath must be able to calculate operate on non-XML data types
• XPath uses a very simple data model
  1. node sets: //img[not(@alt)]
  2. number: count(//img)
  3. string: /descendant::img[3]/@src
  4. boolean: starts-with(/html/@lang, 'en')

XPath Usages

• XPath is used in different technologies
  • XSLT uses XPath as its expression language
  • XSDL uses XPath for selecting identity constraint nodes
  • DOM uses XPath as a way to select DOM nodes
• Depending on the environment, expression must yield certain results
  • for conditionals, a boolean must be returned
  • iterations (in XSLT) only loop over nodes
  • when printing out text, a string must be produced
• XPath has built-in rules for casting types
  • node set → boolean: empty is false, non-empty is true
  • node → string: take the string value (i.e., concatenate all text node descendants)
  • string → number: interpret as decimal notation (otherwise return NaN)
  • XPaths often return surprising results (//a[starts-with(@href, https)])

Function Library

• XPath has a small library of built-in functions
  • useful for basic XPath-level functions
  • other specs are allowed to extend it (XSLT does it)
• XPath functions return results of various data types
  • boolean: boolean, contains, false, lang, not, starts-with, true
  • number: ceiling, count, floor, last, number, position, round, string-length, sum
  • string: concat, local-name, name, namespace-uri, normalize-space, string, substring, substring-after, substring-before, translate
  • node set: id

Using Functions

• Functions and location paths are orthogonal
  • each construct may be based on the other
  • it is possible to nest them arbitrarily
  • predicates often contain functions

    //a[substring(@href,string-length(@href)-2)='pdf']

• XPaths can become powerful and complex
  • writing some code or thinking about an XPath?
  • XPaths are more declarative
  • they may be more robust against changes in the XML schema
  • they can be optimized by a smart XPath implementation

XPath Selects

• Query languages select and recombine
  1. look up all addresses by zip code
  2. for each zip code, count the number of addresses
• XSLT fills in the missing parts (as a programming language)
  • XSLT can construct XML and re-apply XPath
• XQuery fills in the missing parts (query-wise)
  • 80% of XQuery are XPath (in version 2.0, though)
  • the remaining 20% are bindings, constructors, and glue

XPath is Important

• XPath is a basic tool of the XML toolbox
• XPath is reused in various XML technologies
• XPath selects parts of an XML document
• XPath can do more general things by using expressions