Wednesday, October 19, 2011

Consuming XML from REST Services

I spent some time recently developing a process that would bump addresses up against the Google Maps Geocoding service, and the Bing Maps Location service, retrieve and parse the returned XML, and compare the resulting addresses. It had been a while since I last used the XmlDocument, and I re-encountered things I thought best to write down, lest I forget again.

If you would like to see the results yourself, the following URLs were used:
Bing:[address]&output=xml&key=[key] (you’ll need your own key)

Using Web Requests and Web Responses

The first trick to all this is making web requests from your code. I chose to use the HttpWebRequest and HttpWebResponse classes in the System.Net namespace. I created a helper class which would take a URL, generate a GET request to that URL, then process the response stream into a string.

public static string GetWebResponseBody(string RequestUrl)
StringBuilder stringBuilder = new StringBuilder();

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(RequestUrl);
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (Stream stream = response.GetResponseStream())
UTF8Encoding encoding = new UTF8Encoding(false);
byte[] byteBuffer = new byte[8192];
int count = 0;

count = stream.Read(byteBuffer, 0, byteBuffer.Length);

if (count != 0)
stringBuilder.Append(encoding.GetString(byteBuffer, 0, count));
while (count > 0);

return stringBuilder.ToString();

This would give back the XML to be parsed, using the LoadXml() method on the XmlDocument, like so:

string xml = WebHelper.GetWebResponseBody(RequestUrl);
XmlDocument xmlDocument = new XmlDocument();

The first attempt to the Google service worked great. The XML was returned and parsed successfully. When I attempted the Bing service, however, I received this helpful message: Data at the root level is invalid. Line 1, position 1

Long story short, the Bing service emits a UTF-8 identifier at the very first position, which the XmlDocument does not particularly like. The trick, then, is not to grab the web response stream as a string, but feed it directly into the XmlDocument, using the Load() method:

public static XmlDocument GetWebResponseXml(string RequestUrl)
XmlDocument xmlDoc = xmlDoc = new XmlDocument();

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(RequestUrl);
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (Stream stream = response.GetResponseStream())

return xmlDoc;

Now, the following works for both Google and Bing, regardless of that UTF-8 identifier:

XmlDocument xmlDocument = WebHelper.GetWebResponseXml(RequestUrl);


Looking for XML Nodes

Once you have the XmlDocument, you can use the SelectNodes() or SelectSingleNode() methods to find elements, using XPATH statements. However, you must be aware of whether the root element (document element) has a default namespace.

For example, the XML returned by the Google service is simple and straight-forward:

<?xml version="1.0" encoding="UTF-8"?>


There is no default namespace, so the following code works to find the status of the call to the Google service:

status = xmlDocument.DocumentElement.SelectSingleNode("status").InnerText;


Now let’s look at the XML from the Bing service (I’ve stripped it down to the essentials):

<?xml version="1.0" encoding="utf-8"?>
<Response xmlns="">


If we try that same bit of code to find the status of the call to the Bing service, you’ll get a Null Reference Exception:

status = xmlDocument.DocumentElement.SelectSingleNode("StatusDescription").InnerText;

Why? Because we have not taken into account the default namespace. We need to reference the namespace in any XPATH statements we use, in this case. Use the XmlNamespaceManager to declare the namespace information for the XmlDocument, then reference it in the SelectNodes() or SelectSingleNode() methods:

XmlNamespaceManager nsMgr = new XmlNamespaceManager(xmlDocument.NameTable);
nsMgr.AddNamespace("api", "");

status = xmlDocument.DocumentElement.SelectSingleNode("api:StatusDescription", nsMgr).InnerText;


You can see that I declared the “api” alias for the namespace, which we find in the Bing XML. Just remember when doing more complex XPATH, that alias prefixes any node you would then try to find.

No comments:

Post a Comment