Spatial Data Services

Exclusive offer: get 50% off this eBook here
Learning Bing Maps API

Learning Bing Maps API — Save 50%

Obtain geographical data from Bing Maps and display them on the map with this book and ebook.

$20.99    $10.50
by Artan Sinani | November 2013 | Open Source Web Development

In this article by Artan Sinani, author of the book Learning Bing Maps API, iwe will learn geocoding setting up automated jobs on the Bing Maps servers. We will also learn the techniques of monitoring the jobs, and on their completion, parsing the response.

The topics covered in this article are as follows:

  • Geocoding dataflow input data
  • The job
  • Getting job status

(For more resources related to this topic, see here.)

So far we have worked with relatively small sets of data; for larger collections, Bing Maps provide Spatial Data Services. They offer the Data Source Management API to load large datasets into Bing Maps servers, and the Query API to query the data.

In this article, we will use the Geocode Dataflow API, which provides geocoding and reverse geocoding of large datasets. Geocoding is the process of finding geographic coordinates from other geographic data, such as, street addresses or postal codes. Reverse geocoding is the opposite process, where the coordinates are used to find their associated textual locations, such as, addresses and postal codes. Bing Maps implement these processes by creating jobs on Bing Maps servers, and querying them later. All the process can be automated, which is ideal for huge amounts of data.

Please note that strict rules of data usage apply to the Spatial Data Services (please refer http://msdn.microsoft.com/en-us/library/gg585136.aspx for full details). At the moment of writing, a user with a basic account can set up to 5 jobs in a 24-hour period.

Our task in this article is to geocode the addresses of ten of the biggest technology companies in the world, such as Microsoft, Google, Apple, Facebook, and so on, and then display them on the map. The first step is to prepare the file with the companies' addresses.

Geocoding dataflow input data

The input and output data can be supplied in the following formats:

  • XML (content type application/xml)
  • Comma separated values (text/plain)
  • Tab-delimited values (text/plain)
  • Pipe-delimited values (text/plain)
  • Binary (application/octet-stream) used with Blob Service REST API

We will use the XML format, for its clearer declarative structure.

Now, let's open Visual Studio and create a new C# Console project named LBM.Geocoder. We then add a Datafolder, which will contain all the data files and samples with which we'll work in this article, starting with the data.xml file we need to upload to the Spatial Data servers to be geocoded.

<?xml version="1.0" encoding="utf-8"?>
<GeocodeFeed
xmlns="http://schemas.microsoft.com/search/local/2010/5/geocode"
Version="2.0">
<GeocodeEntity Id="001"
xmlns="http://schemas.microsoft.com/search/local/
2010/5/geocode">
<GeocodeRequest Culture="en-US" IncludeNeighborhood="1">
<Address AddressLine="1 Infinite Loop"
AdminDistrict="CA" Locality="Cupertino" PostalCode="95014" />
</GeocodeRequest>
</GeocodeEntity>
<GeocodeEntity Id="002"
xmlns="http://schemas.microsoft.com/search/local/2010/5/
geocode">
<GeocodeRequest Culture="en-US" IncludeNeighborhood="1">
<Address AddressLine="185 Berry St"
AdminDistrict="NY" Locality="New York" PostalCode="10038"
/>
</GeocodeRequest>
</GeocodeEntity>

The listing above is a fragment of that file with the addresses of the companies' headquarters. Please note that the more addressing information we provide to the API, the better quality geocoding we receive. In production, this file would be created programmatically, probably based on an Addresses database. The ID of GeocodeEntity, could also be stored, so that the data is matched easier once fetched from the servers. (You can find the Geocode Dataflow Data Schema, Version 2, at http://msdn.microsoft.com/en-us/library/jj735477.aspx.)

The job

Let's add a Jobclass to our project:

public class Job
{
private readonly string dataFilePath;
public Job(string dataFilePath)
{
this.dataFilePath = dataFilePath;
}
}

The dataFilePath argument is the path to the data.xml file we created earlier.

Creating the job is as easy as calling a REST URL:

public void Create()
{
var uri =
String.Format("{0}?input=xml&output=xml&key={1}",
Settings.DataflowUri, Settings.Key);
var data = File.ReadAllBytes(dataFilePath);
try
{
var wc = new WebClient();
wc.Headers.Add("Content-Type", "application/xml");
var receivedBytes = wc.UploadData(uri, "POST",
data);
ParseJobResponse(receivedBytes);
}
catch (WebException e)
{
var response = (HttpWebResponse)e.Response;
var status = response.StatusCode;
}
}

We place all the API URLs and other settings in the Settings class:

public class Settings
{
public static string Key = "[YOUR BING MAPS KEY];
public static string DataflowUri =
"https://spatial.virtualearth.net/REST/v1/dataflows/
geocode";

public static XNamespace XNamespace =
"http://schemas.microsoft.com/search/local/ws/rest/v1";
public static XNamespace GeocodeFeedXNamespace =
"http://schemas.microsoft.com/search/local/2010/5/geocode";
}

To create the job, we need to build the Dataflow URL template with a Bing Maps Key, and parameters such as input and output formats. We specify the latter to be XML.

Next, we use a WebClient instance to load the data with a POST protocol. Then, we parse the server response:

private void ParseJobResponse(byte[] response)
{
using (var stream = new MemoryStream(response))
{
var xDoc = XDocument.Load(stream);
var job = xDoc.Descendants(Settings.XNamespace +
"DataflowJob").FirstOrDefault();
var linkEl = job.Element(Settings.XNamespace +
"Link");
if (linkEl != null) Link = linkEl.Value;
}
}

Here, we pass the stream created with the bytes received from the server to the XDocument.load method. This produces an XDocument instance, which we will use to extract the data we need. We will apply a similar process throughout the article to parse XML content. Note that the appropriate XNamespace needs to be supplied in order to navigate through the document nodes.

You can find a sample of the response inside the Data folder (jobSetupResponse.xml), which shows that the link to the job created is found under a Link element within the DataflowJob node.

Getting job status

Once we have set up a job, we can store the link on a data store, such as a database, and check for its status later. The data will be available on the Microsoft servers up to 14 days after creation.

Let's see how we can query the job status:

public static JobStatus CheckStatus(string jobUrl)
{
var result = new JobStatus();
var uri = String.Format("{0}?output=xml&key={1}", jobUrl,
Settings.Key);
var xDoc = XDocument.Load(uri);
var job = xDoc.Descendants(Settings.XNamespace +
"DataflowJob").FirstOrDefault();
if (job != null)
{
var linkEls = job.Elements(Settings.XNamespace +
"Link").ToList();
foreach (var linkEl in linkEls)
{
var nameAttr = linkEl.Attribute("name");
if (nameAttr != null)
{
if (nameAttr.Value == "succeeded") result.
SucceededLink = linkEl.Value;
if (nameAttr.Value == "failed") result.FailedLink
= linkEl.Value;
}
}
var statusEl = job.Elements(Settings.XNamespace +
"Status").FirstOrDefault();
if (statusEl != null) result.Status = statusEl.Value;
}
return result;
}

Now, we know that to query a Data API we need to first build the URL template. We do this by attaching the Bing Maps Key and an output parameter to the job link.

The response we get from the server, stores the job status within a Link element of a DataflowJob node (the jobResponse.xml file inside the Data folder contains an example). The link we need has a name attribute with the value succeeded.

Summary

When it comes to large amounts of data, the Spatial Data Services offer a number of interfaces to store, and query user data; geocode addresses or reverse geocode geographical coordinates. The services perform these tasks by means of background jobs, which can be set up and queried through REST URLs.

Resources for Article:


Further resources on this subject:


Learning Bing Maps API Obtain geographical data from Bing Maps and display them on the map with this book and ebook.
Published: November 2013
eBook Price: $20.99
Book Price: $34.99
See more
Select your format and quantity:

About the Author :


Artan Sinani

Artan Sinani is a Web Developer living between London and a small town in the north-west of Spain.

Currently, he works as a Senior Developer at Local Data Company, where he uses his passion for good design and clean programming to build beautiful interfaces, including geospatial data visualizations.

He hacks open source code at github.com/artisinani and blogs at lugolabs.com.

Books From Packt


OpenStreetMap
OpenStreetMap

Instant OpenLayers Starter [Instant]
Instant OpenLayers Starter [Instant]

Windows Phone 7.5: Building Location-aware Applications
Windows Phone 7.5: Building Location-aware Applications

Microsoft SharePoint 2010 Enterprise Applications on Windows Phone 7
Microsoft SharePoint 2010 Enterprise Applications on Windows Phone 7

 Windows Phone 7.5 Data Cookbook
Windows Phone 7.5 Data Cookbook

 OpenLayers 2.10 Beginner's Guide
OpenLayers 2.10 Beginner's Guide

Mind Mapping with FreeMind
Mind Mapping with FreeMind

Instant Heat Maps in R How-to [Instant]
Instant Heat Maps in R How-to [Instant]


No votes yet

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
9
E
H
x
F
2
Enter the code without spaces and pay attention to upper/lower case.
Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software