Checking OpenStreetMap Data for Problems

Exclusive offer: get 50% off this eBook here
OpenStreetMap

OpenStreetMap — Save 50%

Be your own cartographer

$23.99    $12.00
by Jonathan Bennett | September 2010 | Open Source

OpenStreetMap is a diverse project with hundreds of thousands of people contributing data and making use of it in different ways. As a result, many of the resources that mappers have created and use are scattered around the Internet, but the project data and much of the documentation is hosted at openstreetmap.org, on servers operated by the OpenStreetMap Foundation.

In this article by Jonathan Bennett, author of the book OpenStreetMap, we'll look at some of the following tools you can use to check OpenStreetMap data in a particular area, and what problems they can and can't tell you about:

  • The data inspection tools on openstreetmap.org
  • The NoName map
  • ITOWorld OSM Mapper
  • Geofabrik's OSM Inspector

Some of these tools identify specific problems, while others allow you to see when changes to the data were made and by whom. In either case, all these tools can do is offer guidance, not black-and-white answers, and you always need to use judgment when checking data.

 

OpenStreetMap

OpenStreetMap

Be your own cartographer

  • Collect data for the area you want to map with this OpenStreetMap book and eBook
  • Create your own custom maps to print or use online following our proven tutorials
  • Collaborate with other OpenStreetMap contributors to improve the map data
  • Learn how OpenStreetMap works and why it's different to other sources of geographical information with this professional guide

Read more about this book

(For more resources on OpenStreetMap, see here.)

It's important to remember that there are few fixed ideas of what is "wrong" data in OpenStreetMap. It should certainly be an accurate representation of the real world, but that's not something an automatic data-checking tool can detect. There may be typographical errors in tags that prevent them from being recognized, but there are also undocumented tags that may accurately describe a feature, yet be unknown to anyone except the mapper who used them. The latter is fine, but the former is a problem.

It's tempting to use the two map renderings on openstreetmap.org as a debugging tool, but this can be misleading. Not every possible feature is rendered, and many problems with the data, such as duplicate nodes or unjoined ways, won't be obvious from a rendered map. If a feature you've mapped doesn't render when a similarly tagged one does, there's an issue, but a feature appearing in the map doesn't mean it's free of problems, and a feature that doesn't appear isn't necessarily wrong.

Ultimately, you will have to use your own judgment to find out whether or not an issue reported by one of these tools is really an error in the data. You can always contact other members of the OpenStreetMap community.

This is only a selection of the more widely used quality assurance tools used by mappers. For a more complete list, refer to http://wiki.openstreetmap.org/wiki/Quality_Assurance.

Inspecting data with openstreetmap.org's data overlay and browser

The openstreetmap.org website has a range of tools you can use to inspect the data in the database, both current and past. Some of the tools aren't obvious from the front page of the site, but are easily found if you know where they are.

The tools, which consist of the data map overlay and the data browser pages, allow you to see the details of any object in the OpenStreetMap database, including coordinates, tags, and editing history, without the need to launch an editor or read raw XML. As these tools work directly with the data in the OpenStreetMap database, they always show the most up-to-date information available. However, they simply provide raw information, and don't provide any guidance on whether the geometry or tagging of any feature could be problematic.

The easiest way of inspecting data is to start with the data map overlay. Go to the map view and find Compton (or any other area you want to inspect). Open the layer chooser by clicking on the + sign at the top-right. Click the checkbox labeled Data, and a box will appear to the left of the map view. After a short delay, the data overlay will appear, and a list of objects will appear in the box.

JavaScript speed and the OpenStreetMap data overlay
The data overlay and the accompanying list of objects make heavy use of JavaScript in your browser, and depending on how many objects are currently in your map view, can use a lot of processing power and memory. Some older browsers may struggle to even show the data overlay. Mappers have reported that Firefox 3.5, Apple Safari, and Google Chrome all work well with the data overlay, even with large numbers of objects.

Once the data for the area you're inspecting has loaded, you'll see something like the following image:

In the preceding image, on the left you can see the Object list, which gives a text description of every feature in the current map view, giving its primitive type and either its ID number or a name, if the feature has one. On the right is the map with the data overlay, which highlights every feature in the current area, whether they're rendered on the map or not. This last point is worth repeating: Not every type of feature gets rendered on the two map renderings used on openstreetmap.org, and those that do can take some time to appear if the load on the rendering engines is high. Any feature in the database will always appear in the data overlay.

Inspecting a single feature

To inspect an individual feature, either click on its entry in the object list, or on its highlight in the map view. Both the object list and the overlay will change to reflect this. Occasionally, an area feature may get drawn on top of other features, preventing you from selecting the ones underneath, but you'll still be able to select them from the list.

Let's select The Street and inspect its data. Either click on its name in the object list, or on the way in the map view, and the object list should change to show the tags applied to the feature, and you should see something like the following in the object list:

This gives a list of the tags attached to the feature. If you click on Show History, a list of the edits made to the current feature is added to the list. To get more information, click on the Details link next to the feature's name, and you'll be taken to the data browser page for that object, as follows:

Here you see far more details about the feature we're inspecting. Apart from the object ID and its name, you can find the time when the object was last edited and by whom, and in which changeset. There are clickable links to any related objects and a map showing the feature's location.

At the bottom of the page are links to the raw XML of the feature, the history page of the feature, and a link to launch Potlatch—the online editor—for the area surrounding the feature.

Checking a feature's editing history

The OpenStreetMap database keeps every version of every feature created, so you can inspect previous versions and see when and how a feature has changed. To look at a feature's history, click on the link at the bottom of its data browser page. For the Watts Gallery in Compton, you should see something like the following:

You can see each version of the object listed in full, including which mapper created that version in which changeset, and what the tags for that version were. There's currently no way of showing any previous version or the changes between versions on the map, but third-party tools such as OSM Mapper provide some of these features.

Inspecting changesets

Along with looking at individual features, you can see how the map gets changed by looking at changesets. Since version 0.6 of the OpenStreetMap API went live in April 2009, every change to the map has to be part of a changeset. A changeset is a list of related edits made to OpenStreetMap data, with its own set of tags. What goes into a changeset is entirely up to the mapper creating it.

You can view the list of recent changesets by clicking on the History tab at the top of the map view. This will show a list of the 20 most recent changesets whose bounding box intersects your current map view. Note that this doesn't guarantee that any changesets listed include any edits in your current view, and any changesets covering a large area will be marked with (big) in the list.

OpenStreetMap Be your own cartographer
Published: September 2010
eBook Price: $23.99
Book Price: $39.99
See more
Select your format and quantity:

Read more about this book

(For more resources on OpenStreetMap, see here.)

Finding unsurveyed areas with the NoName layer

The NoName layer is a complete rendering of OpenStreetMap designed to highlight the urban areas that may need attention. To view the NoName layer, open the layer chooser in the map view on openstreetmap.org and choose NoName from the list. In NoName, any way tagged as a road but without a corresponding name=* tag is highlighted in red, as seen in the following image. The NoName layer was created to show areas where streets had been traced from aerial imagery, but no follow-up survey had been done to collect the street names.

Large numbers of adjacent streets highlighted in the NoName layer probably means that the area needs surveying on the ground, to collect the street names and other points of interest. However, a road or street not having a name in OpenStreetMap isn't necessarily an error: The street may not actually have a name, or at least not one that can be identified by surveying it. You shouldn't tag a road with a blank name or some other incorrect value just to make it disappear from the NoName layer.

The NoName layer is provided by CloudMade, which also provides the layer as a downloadable map for Garmin devices at http://downloads.cloudmade.com/.

OSM Mapper

OSM Mapper (http://www.itoworld.com/product/osm/) is a web-based tool provided by a British transport intelligence firm, ITO, and provides similar facilities to those on the openstreetmap.org website, but allows you to sort and filter the data for a given area. At present, OSM Mapper only processes ways, not nodes or relations, but is still useful for finding sets of data with particular tags, and edited by particular mappers.

You need to register at the ITO website to use OSM Mapper, including supplying an e-mail address, and responding to a confirmation e-mail. You won't receive any further e-mail from ITO after registering. The data in OSM Mapper is usually updated once per day, so any edits you make won't show up here immediately.

Creating an area to analyze

To look at the data using OSM Mapper, you need to define one or more areas that interest you. The size of an area is limited to prevent overloading the system, although the limit of around 40 miles across is still quite large. OSM Mapper is most useful on much smaller areas than this, such as a single town. When you first register, you'll be shown an example area of Ipswich in the UK. To create your own area, click on the outline arrow to the right of the area chooser in the top right-hand corner of the page, and choose Create a new area. Use the slippy map that appears to choose an area, give it a name, and click Save.

Filtering and sorting data

Once you've defined an area, you can start to analyze its data. OSM Mapper shows all ways as outlines, which it color-codes according to the filters you create. To add a filter, choose from the options listed to the right of the map viewer: Tags, Sessions, or Users. These filter the current view according to the tags applied to the ways, by editing "session" or depending on which mapper last edited them respectively.

A session in OSM Mapper refers to a group of edits by a single mapper over a short period of time. OSM Mapper predates the use of changesets in OpenStreetMap, and there may or may not be a correlation between an OpenStreetMap changeset and an OSM Mapper session.

Click on Tags, and a table showing all keys that appear on ways in your current area will appear, complete with usage counts for each one. You can click on the headers in this table to sort the values by that column. If you now click on highway in the list of keys, you should see something like the following image:

Any ways that don't have a highway=* tag have been faded out, and the remaining ways have been color-coded according to the value of highway=* used. The list has also changed to show those values and their usage counts. Clicking on a value will filter the view further to just the ways with that tag, and the list will change to show each one, listed by name if it has one, or ID if not. Click on an individual way and it will be highlighted, and its details will be shown in the list, including links that lead back to the data inspection tools on openstreetmap.org.

Creating a view

OSM Mapper allows you to create more complex combinations of filters than we've seen so far, and store them as Views so you can reuse them. Click on the + sign next to the View drop-down to the right of the map, and a box will appear. This lists the color table in use and any filters that apply to your current map view.

Let's create a view. Go back to the list of ways tagged as highway=*. To add a filter, use the arrow to the right of the current filter list. Click on this and you will get two filter options, to filter by the current tag being present or not present, so choose Add: Tag Present: highway to add this filter. You'll be taken back to the options, but you should now have a filter of Tag present: highway listed. Now click on Users, then Jonathan Bennett, and click on the filter drop-down and choose Add: User=Jonathan Bennett. You'll again be taken back to the filter options. If you click through to the list of tags again, you should now see something like the following screenshot:

You can see the filters currently applied to the data, along with the table of usage numbers. These numbers represent the results of the filters, rather than the whole area. You can save the view by clicking the clear down-arrow to the right of the View name and choosing Save the current view.

You could add further filters to the view, such as a filter for roads with no name=* tag present, which would show you any street you may need to resurvey for a name, for example. Note that you can only filter to include a particular session, not exclude one.

OSM Inspector

OSM Inspector (http://tools.geofabrik.de/osmi/) is a web-based tool provided by German consultancy GeoFabrik, and it checks for several classes of potential problems with OpenStreetMap data. The interface looks as shown in the following screenshot:

OSM Inspector doesn't require any registration or configuration; simply visit the site and use the slippy map to find the area in which you're interested. It's updated once every two days, so again your edits won't show up straight away. The date of the current data in use is shown below the slippy map. Across the top of the page are the view controls: a View drop-down list that chooses which class of problem to look at, the Base layer drop-down that allows you to choose which map rendering to use, and a transparency slider for the slippy map.

OSM Inspector has four Views that work for the whole world:

  • Geometry tells you about problems such as self-intersecting ways, duplicate nodes in ways, and very long ways
  • Tagging tells you about possible typographical errors in tags or other problematic keys and values
  • Places identifies settlements and highlights, unnamed places, and problems with population tags
  • Highways shows problems such as unnamed roads or ones without a reference number

The remaining four views are as follows:

  • Addresses looks for problems in addressing information that has been added to buildings and streets
  • Boundaries shows administrative boundaries such as state and county borders
  • Water shows any type of waterway, and problems such as unnamed rivers
  • Five public transport-related views show details of railways, bus routes, ferries, and other public transport infrastructure

There are also several country-specific views that have been used by mappers importing data in those countries; you won't need to use these.

Choose any one of these views, and a list of tests appears in the left-hand column. Each test has a checkbox, allowing you to turn it on and off in the slippy map, and a key for how it's represented on the map. Tests are also arranged in groups, and you can turn whole groups on and off using its checkbox. Hovering your mouse pointer over a test name will pop up a description of what that test checks for, and which zoom levels it shows up at.

Clicking on any highlighted feature in the slippy map will show its details in the right-hand column, as follows:

Rather than showing the tags applied to the selected object, this panel shows information relating to the problem OSM Inspector has identified with the feature. In the preceding screenshot, you can see that Compton is tagged as a village (place=village), but has no corresponding population=* tag. There are four icons near the top of the pane:

  • The magnifying glass zooms the map to the selected feature
  • The icon with a P launches Potlatch centered on the selected feature
  • The icon with a J will center the current view in JOSM on the selected feature, provided JOSM is already running and has the Remote Control plugin installed
  • The map and magnifying glass icon opens the openstreetmap.org data browser for the selected object

Below the Selection pane is the Data pane, which will show a list of all objects matching one of the tests in the current view. Click on the magnifying glass icon next to a feature in this list in to zoom to it.

OSM Inspector is very well documented in its own online help and in the OpenStreetMap wiki. Click on the Help link at the top of the page to access the online help.

Summary

Mistakes in data are inevitable, but that doesn't mean you can't correct them. It should be possible to find the cause of any problems you're having with OpenStreetMap data using one or more of the tools we've looked at:

  • The data inspection tools on openstreetmap.org
  • The NoName layer
  • ITOWorld OSM Mapper
  • Geofabrik's OSM Inspector

However, these tools can only provide guidance, and you should always use common sense when interpreting their results. Remember to ask the OpenStreetMap community if you're in any doubt about a possible error in the data.


Further resources on this subject:


OpenStreetMap Be your own cartographer
Published: September 2010
eBook Price: $23.99
Book Price: $39.99
See more
Select your format and quantity:

About the Author :


Jonathan Bennett

Jonathan Bennett is a British journalist, writer, and developer. He has been involved in the OpenStreetMap project since 2006, and is a member of the OpenStreetMap Foundation. He has written for print and online technical publications including PC Magazine, ZDNet, CNET, and has appeared on television and radio as a technology commentator. He has an extensive collection of out-of-date printed maps.

Books From Packt


Drupal 7
Drupal 7

jQuery 1.4 Reference Guide
jQuery 1.4 Reference Guide

Moodle 2.0 First Look
Moodle 2.0 First Look

Joomla! Social Networking with JomSocial
Joomla! Social Networking with JomSocial

Apache OfBiz Cookbook
Apache OfBiz Cookbook

Plone 3.3 Site Administration
Plone 3.3 Site Administration

Nginx HTTP Server
Nginx HTTP Server

IT Inventory and Resource Management with OCS Inventory NG 1.02
IT Inventory and Resource Management with OCS Inventory NG 1.02


No votes yet

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
y
K
w
A
p
k
Enter the code without spaces and pay attention to upper/lower case.
Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software