Using XPath to Extract Data From XML...

The following article is going to demonstrate how to use the XML Path (XPath) Language to extract data from an XML document. This article will introduce you to what XPath is, how the XPath recommendation is implemented in the Microsoft.NET framework, and finally, examine a scenario where a web application relies on XPath to extract data from an XmlDocument object.

By Tyler Davey Date: July 22, 2004 Download the code.

Introduction

The following article is going to demonstrate how to use the XML Path (XPath) Language to extract data from an XML document. This article will introduce you to what XPath is, how the XPath recommendation is implemented in the Microsoft.NET framework, and finally, examine a scenario where a web application relies on XPath to extract data from an XmlDocument object.

This article assumes that you have basic knowledge in XML. It is also recommended that you have knowledge in the System.Xml.XmlReader object in Microsoft, in order to compare how XPath works against the XmlReader (which is based on the DOM). Familiarity with creating web applications in Visual Studio .net is strongly recommended.

XPath – The Path To Nodes

The title sums up what XPath is: an expression to return a node result set to the calling method or application. A node is a complete element within an XML document. For instance, if you have an XML Document that looks like this:

<customers>
  <customer id="1">
    <companyName>DotNetJohn</companyName>
  </customer>
  <customer id="2">
    <companyName>Microsoft</companyName>
  </customer>
</customers>

A node could be
<companyName>DotNetJohn</companyName>
or
DotNetJohn
or
id="1"

The calling method or application specifies an XPath expression, and XPath returns a single or group of nodes that satisifies the expression. Think of XPath as a query language - by specifying a statement, XPath will retrieve whatever data you request in a given XML document, which is very similar to a select clause in SQL.

Let's take a look at a sample XPath statement. If we were to execute the following XPath query:

  /customers/firstName

XPath would traverse down an XML document looking first for the <customers> node. Once found, it will search inside of the customers node for the <firstName> node. Once it reaches the firstName node, XPath will return all of the nodes back to the caller.

XPath was introduced on November 16th 1999 as an addition to the W3C XSLT (XML Stylesheet Language/ Transformations) recommendation. XPath was to be used for querying data contained within an XML document to participate within an XSLT instance. However, coding languages, such as .NET, have taken the XPath recommendation and used it within their class libraries to not only satisify the recommendation to the W3C specification, but also as an alternative to the DOM for iterating over an XML document. And thank goodness they did. XPath not only gives users much more flexibility, but it is also a lot more lightweight and easier to use.

The major characters involved in an XPath query are as follows:

In the above chart, I showed you how you could traverse an XML document by using the / or // characters. Both of these characters deal with absolute path definitions. That is to say, they know where they are starting from and where they want to go to. But what if you needed a relative path? For instance, say you needed to traverse up the XML document from a child element, to find out who its' parents are? This is performed by using the method localization. Localization combines axis names, node tests> and predicates to form a query based on relative or unknown paths.

An axis name specifies what you are searching for based on the node you are providing. The available axis names are as follows:

The node test is a node or wildcard character or both. Following the node test, there can be zero or more predicates. A predicate allows you to do more refined searches based on the node test. The syntax of a relative path search is as follows:

  axis::nodetest[predicates]

Where [predicates] are optional. For example:

  parent::firstName

would return the parents of the element firstName (customers).

  parent::firstName[attribute::name='Tyler']

would return the parents of the element firstName that has an attribute containing the value "Tyler".

XPath also includes support for expressions and functions. Expressions can be mathematical (+,-,div,*,mod), equality(=,!=), comparison (<, <=, >, >=) or boolean (or, and). Functions allow you to perform dynamic operations to convert or manipulate the data that gets returned. There are libraries for functions to manipulate node sets, strings, numbers, and booleans. I recommend looking them up at http://www.w3c.org/TR/xpath#corelib

XPath – Microsoft's Implementation

Now that you understand the syntax of XPath, let's look at how the .NET framework has implemented XPath. Microsoft has specified a namespace in its class library called System.Xml.XPath. There are several classes, enumerations and an interface that exist inside of this namespace, however, we will focus on four of the key classes – XPathDocument, XPathExpression, XPathNavigator, and XPathNodeIterator. Each example we will do for these classes will be based on the following XML document:

<customers>
  <customer id="1">
    <companyName>DotNetJohn</companyName>
  </customer>
  <customer id="2">
    <companyName>Microsoft</companyName>
  </customer>
</customers>

The XPathDocument class provides your code with a fast, read-only cache for processing XSLT statements against an XML document. The class on its own is fairly useless in an XPath environment. However, if you execute its rimary method, CreateNavigator(), you will be returned an instance of the XPathNavigator class. This class is key to using XPath, as it allows you to traverse along the XML document, going both forwards and backwards. The following example creates a new instance of an XPathDocument to create an XPathNaviagtor object to gain access to the nodes contained within the above xml document.

The XPathExpression class allows you to take an XPathNavigator object and execute expressions against it. Beyond just executing expressions, this class also has functionality to return the data type of the result set and to sort a result set. Following the above example, if we wanted to get access to all the customer elements, we could write the following expression:

When you use the XPathNavigator to select a group of nodes, those nodes get stored inside of an XPathNodeIterator object. Once inside that object, because it extends the ICloneable interface, your iteration object of nodes can be walked over using the MoveNext() method. To complete our example, we can add the XPathNodeIterator object to get all of the names of our customers and display them in a drop down list:

The XPath language is not limited to just those object implementations either. XPath has been implemented in the System.Xml.XmlNode class as well, allowing the methods SelectNodes() and SelectSingleNode() to specify an XPath query as their parameter. This is key, as the XmlNode class is contained in other Xml classes, such as System.Xml.XmlElement and System.Xml.XmlDocument. Since XmlDocument is a class that allows you to load an XML document, this means that most of your XML code that you have written to date probably has access to perform queries using XPath without making any drastic changes. The following example uses the XmlDocument class and XPath together to create a dynamic web site.

The Language Selector Example

We are going to create a web site that changes the content from one language to another, based on a users selection. It will use the following XML document:

<?xml version="1.0" encoding="utf-8" ?>
<pageSettings size="10" fontFace="Verdana">
  <english>
    <lblFirstName>First Name:</lblFirstName>
    <lblLastName>LastName:</lblLastName>
    <ddlLanguage>
      <item0>English</item0>
      <item1>French</item1>
      <item2>Spanish</item2>
    </ddlLanguage>
    <btnDisplay>Display</btnDisplay>
    <lblShowMessage>Welcome to my little site, </lblShowMessage>
  </english>
  <french>
    <lblFirstName>Prénom</lblFirstName>
    <lblLastName>Dernier Nom:</lblLastName>
    <ddlLanguage>
      <item0>Anglais</item0>
      <item1>Français</item1>
      <item2>Espagnol</item2>
    </ddlLanguage>
    <btnDisplay>Affichage</btnDisplay>
    <lblShowMessage>Bienvenue à mon petit emplacement, </lblShowMessage>
  </french>
  <spanish>
    <lblFirstName>Nombre:</lblFirstName>
    <lblLastName>Nombre Pasado:</lblLastName>
    <ddlLanguage>
      <item0>Inglés</item0>
      <item1>Francés</item1>
      <item2>Español</item2>
    </ddlLanguage>
    <btnDisplay>Exhibición</btnDisplay>
    <lblShowMessage>Recepción a mi pequeño sitio, </lblShowMessage>
  </spanish>
</pageSettings>

To begin, open up Microsoft Visual Studio .net and create a new web application project. Name the application LanguageSelector, and open the WebForm1.aspx page. Create the web page with the following html code: Note – you can use the toolbox to drag and drop the items onto the web page, but for the sake of clarity, I think that showing you what the resulting html code will look like in the designer will be easier from a coding perspective.

The form we have created above is very simple: two textboxes to accept a First and Last Name, a button to cause an action to occur to display a message in our label, lblDisplayMessage. The only part really to take notice to is the drop down list. If you notice, I’ve set the postback action to true and there are three different options, representing a different language to be displayed on the form, pending the users selection.

To represent each language that the user can select, I am going to add the following enumeration object and shared global variable to the code behind window:

Enumerations are integral types that hold specific values. You can create enumerations of just about anything, and the key advantage is that it restricts values that are submitted into objects. In our sample, we only want to support 3 languages, English, French and Spanish. If we want to support Italian, we would have to add it to our enumeration object. However, if we do not support Italian, and a calling method tries to set the variable langSelected to Italian, the application will not compile.

In the Page_Load event we are going to add the following code to instantiate our XmlDocument and load the data for the screen to display properly:

An intelligent question at this point is usually raised: "Why are you using shared (static) variables for your XmlDocument object and LanguageSelected object". We could make these objects not static, but our performance would degrade and we would require more coding to maintain state.

In this application, our XML document is not changing, or at least, it is rarely changing to necessitate a re-read of the document each time a user requests a new language. If we were to change the langSettings to a non static variable, we would have to read in the XML document on every postback. This means, each time the user requests a different language, the application must open a new stream to the XML document, read it back into memory, and discard it after it is done processing it. By keeping the object static, the object will exist for the lifetime of the page scope. (I could go on about application scope, session scope and page scope, with relation to static and non static variables, but that is ironically outside of the scope of this document). As for keeping the langSelected variable static, as with our langSettings object, we would have to reset its' value on each postback. If the postback is instantiated from the drop down changing its' selected index, there is no impact to our code. However, once our button is clicked, we would have to write code to set the langSelected property inside of the buttons' click event handler. By keeping the object shared, we reduce the number of times we need to set the langSelected property, limiting it to being set only when the user wants it to be set.

The rest of the code in the Page_Load method is fairly simple: we create a new instance of an XmlDocument object, load the LanguageSettings.xml file into the XmlDocument object, and default the langSelected object to English. The only interesting line of code is our call to the LoadLanguage method. This method is going to handle displaying the approriate language for each label and our button on the screen.

I want you to understand what the SelectSingleNode() method is doing. SelectSingleNode() takes an XPath expression, and returns an XmlNode object that contains the result set of a single node from the XPath expression. The node will include the element name, its attributes and all associated values. We could have substituted this entire block of code and performed a SelectNodes() method, which also takes an XPath expression. This would have returned to us a collection of XmlNodes which we could then iterate over to get access to our data.

If we analyze more closely the first line,

lblFName.Text = langSettings.SelectSingleNode("pageSettings/" + langSelected.ToString().ToLower() + "/lblFirstName").InnerText

the string passed into the SelectSingleNode method will evaluate to:

  "pageSettings/english/lblFirstName"

This will cause the XPathNavigator to kick in, iterating over the document. First it will find the pageSettings node, followed by the english node contained within the pageSettings node. Finally, it will search for the lblFirstName node. If it finds it, it will return the entire lblFirstName node back to the caller (langSettings). You may be tempted to use the property Value at first. Value, however, does not return the value contained between the two XML element tags. Rather, Value returns the element name, as the node is the object, and its' value is its' element name.

The line below that piece of code adds an interesting twist:

  lblFName.Font.Name = langSettings.SelectSingleNode("pageSettings").Attributes("fontFace").Value

Because we are returned the XmlNode for pageSettings, we gain access to the collection of Attributes. We can specify the attribute name, and gain access to its' value. This is where Microsoft makes things confusing. If InnerText is used to extract the data between tags for a node, why is Value used to extract the data for an attribute? Well, if you think about it, as I mentioned before, InnerText is the text between two items, in this case, the start and close tags. Value, on the other hand, represents the value of an item. An attribute is defined as a key-value pair, with the key being the attribute name and the value being its' value. A little confusing, but once you get your head around it, it makes sense.

If we run the application at this point, you should be able to see the screen displayed in English. If you try selecting a different language, however, nothing happens. This is because we have not handled the SelectedIndexChanged event for our drop down list. If you double click on the drop down box from the designer, you will be brought into its event handler in the code behind. Add the following code to the code behind file:

Since the enumeration, LanguageSelected, gives us the ability to control which language options are available and we programmed the drop down list so that item 0 (remember, drop down lists items are indexed via 0 base) is English, 1 is French and 2 is Spanish, we just need to set our langSelected to the approriate enumeration value and call the LoadLanguage method. Now, if you run the application again, you should have a system that allows you to change languages on the fly.

Finally, we just need to implement the button so that it displays our message to the user:

Conclusion

This article has hopefully given you a strong sampling of the flexibility of XPath. While the sample is straightforward, more complex problems can be solved using XPath and can drastically reduce the amount of coding you need to do to get your job done, while increasing the performance of your application over using the standard XmlReader object.

Good luck, and happy coding.

Resources:

http://www.msdn.com
http://www.w3c.org

You may download the code here. The download contains both VB and C# versions of the code.