Home Programming QGIS Blueprints

QGIS Blueprints

By Ben Mearns
books-svg-icon Book
Subscription FREE
eBook + Subscription €14.99
eBook €32.99
Print + eBook €41.99
READ FOR FREE Free Trial for 7 days. €14.99 p/m after trial. Cancel Anytime! BUY NOW BUY NOW BUY NOW
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
READ FOR FREE Free Trial for 7 days. €14.99 p/m after trial. Cancel Anytime! BUY NOW BUY NOW BUY NOW
Subscription FREE
eBook + Subscription €14.99
eBook €32.99
Print + eBook €41.99
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
  1. Free Chapter
    Exploring Places – from Concept to Interface
About this book
Publication date:
September 2015
Publisher
Packt
Pages
258
ISBN
9781785289071

 

Chapter 1. Exploring Places – from Concept to Interface

How do we turn our idea into a location-based web application? If you've heard this question before or asked it yourself, you would know that this deceptively simple question can have answers posed in a limitless number of ways. In this book, we will consider the application of QGIS through specific use cases selected for their general applicability. There's a good chance that the blueprint given here will shed some light on this question and its solution for your application.

In this book, you will learn how to leverage this ecosystem, let the existing software do the heavy lifting, and build the web mapping application that serves your needs. When integrated software is seamlessly available in QGIS, it's great! When it isn't, we'll look at how to pull it in.

In this chapter, we will look at how data can be acquired from a variety of sources and formats and visualized through QGIS. We will focus on the creation of the part of our application that is relatively static: the basemap. We will use the data focused on a US city, Newark, Delaware. A collection of data, such as historical temperature by area, point data by address, and historical map images, could be used for a digital humanities project, for example, if one wanted to look at the historical evidence for lower temperatures observed in a certain part of a city.

In this chapter, we will cover the following topics:

  • The software

  • Extract, Transfer, and Load

  • Georeference

  • The table join

  • Geocoding

  • Orthorectification

  • The spatial reference manipulation

  • The spatial reference assignment

  • Projection

  • Transformation

  • The basemap creation and configuration

  • Layer scale dependency

  • Labeling

  • The tile creation

 

The software


QGIS is not a black box—it is a part of a dynamic community of software users and developers. Although this book strives to apply generalizable blueprints to a range of actual web applications, there are times when a deeper understanding of the makeup of the QGIS platform is invaluable; sometimes even as soon as at installation. The latest version of QGIS is 2.10.

The development community and dependencies

As QGIS is open source, no one entity owns the project; it's supported by a well-established community. The project is guided by the QGIS Project Steering Committee (PSC), which selects managers to oversee various areas of development, testing, packaging, and other infrastructure to keep the project going. The Open Source Geospatial Foundation (OSGeo) is a major contributor to software development, and QGIS is considered an official OSGeo project. Many of QGIS' dependencies and complimentary software are also OSGeo projects, and this collective status has served to bring some integration into what can be considered a platform. The Open GIS Consortium (OGC) deliberates and sets standards for the data and metadata formats. QGIS supports a range of OGC standards—from web services to data formats.

When QGIS is at its best, this rich platform provides a seamless functionality, with an ecosystem of open or simply available software ready to be tapped. At other times, the underlying dependencies and ecosystem software require more attention. Since it's an open source software, contributions are always being made, and you have the option of making customizations in code and even contributing to it!

Data format read/write

The OSGeo ecosystem provides capabilities for data format read/write through the OGR Simple Features Library (OGR, originally for OpenGIS Simple Features Reference Implementation) and Geospatial Data Abstraction Library (GDAL) libraries, which support around 220 formats.

Geospatial coordinate transformation

The models of the earth, which the coordinates refer to, are collectively known as Coordinate Reference Systems (CRSs). The spatial reference transformation between systems and projection—from a system in linear versus the one in angular coordinates—is supported by the PROJ.4 library with around 2,700 systems. These are expressed in a plain text format defined by PROJ.4 as Well Known Text (WKT). PROJ.4 WKT is actually very readable, containing the sort of information that would be familiar to the students of cartographic projection, such as meridians, spheroids, and so on.

Analysis

Analysis, or application of algorithmic functions to data is rarely handled seamlessly by QGIS. More often, it is an extension of one of the dependencies already listed before or is provided by System for Automated Geoscientific Analyses (SAGA). Many other analytical operations are provided by numerous QGIS Python plugins.

In general, these libraries will seamlessly transform to or from the formats that we require. However, in some cases, additional dependencies will need to be acquired and either be built and configured themselves or have the code built around them.

Web publishing

QGIS has the capability of publishing to web hosts through both integrated and less immediate means.

Installation

OSGeo project binaries have sometimes been bundled to ease the installation process, given the multitude of interdependencies among projects. Tutorials in this book are written based on an installation using the QGIS standalone installer for Windows.

Linux

QGIS hosts repositories with the most current versions for Debian/Ubuntu and bundled packages for other major Linux distributions; however, these repositories are generally many versions behind. You will find that this is often the case even with the extra repositories for your distribution (for example, EPEL for RHEL flavors). Seeking out other repositories is worthwhile. Another option, of course, is to attempt to build it from scratch; however, this can be very difficult.

Mac

There is no bundled package installer for Mac OS, though you should be able to install QGIS with only one or two additional installations from the binaries readily available on the Web—the KyngChaos Wiki has long been the go-to source for this.

Windows

Installation with Windows is simpler than with other platforms at this time. The most recent version of QGIS, with basic dependencies such as GDAL, is installed with a typical executable installer: the "standalone" installer. In addition, the OSGeo4W (OSGeo for Windows) package installer is very useful for the extended dependencies. You will likely find that beyond simply installing QGIS, you will return to this installer to add additional software to extend QGIS into its ecosystem. You can launch the installer from the Setup shortcut under the QGIS submenu in the Windows Start menu.

OSGeo-Live

The most extensive incarnation of the OSGeo software is embodied in OSGeo-Live, a Lubuntu Virtual Machine (VM) on which all of the OSGeo software is already installed. It is listed here separately since it will boot into its own OS, independent of the host platform.

Updates to OSGeo Live are typically released in tandem with FOSS4G, an annual global event hosted by OSGeo since 2006. Given that these events occur less regularly and are out of sync with OSGeo software development, bundled versions are usually a few releases behind. Still, OSGeo-Live is a quick way to get started.

Now that you've prepared your local machine, let's return to the idea of the generalizable web applications that will be the focus of this book. There are a few elements that we can identify in the process of developing web-mapping applications.

 

Acquiring data for geospatial applications


After any preliminary planning—a step that should include careful consideration of at least the use cases for our application—we must acquire data. Acquisition involves not only the physical transfer of the data, but also processing the data to a particular format and importing it into whatever data storage scheme we have developed. This is usually called Extract, Transform, and Load (ETL).

Though ETL is the first major step in developing a web application, it should not be taken lightly. As with any information-based project, data often comes to us in a form that's not immediately useable—whether because of nonuniform formatting, uncertain metadata, or unknown field mapping. Although any of these can affect a GIS project, as GISs are organized around cartographic coordinate systems, the principle concern is usually that data must be spatially described in a uniform way, namely by a single CRS, as referred to earlier. To that end, data often requires georeferencing and spatial reference manipulation.

For certain datasets, an ETL workflow is unnecessary because the data is already provided via web services. Using hosted data stored on the remote server and read directly from the Web by your application is a very attractive option, purely for ease of development if nothing else. However, you'll probably need to change the CRS, and possibly other formatting, of your local data to match that of the hosted data since hosted services are rarely provided in multiple CRSs. You must also consider whether the hosted data provides capabilities that support the interface of your application. You will find more information on this topic under the operational layer section of this chapter.

Producing geospatial data with georeferencing

By georeferencing, or attaching our data to coordinates, we assert the geographic location of each object in our data. Once our data is georeferenced, we can call it geospatial. Georeferencing is done according to the fields in the data and those available in some geospatial reference source.

The simplest example is when a data field actually matches a field in some existing geospatial data. This data field is often an ID number or name. This kind of georeferencing is called a table join.

Table join

In this example, we will take a look at a table join with some temperature data from an unknown source and census tract boundaries from the US Census. Census' TIGER/Line files are generally the first places to look for U.S. national boundary files of all sorts, not just census tabulation areas.

The temperature data to be georeferenced through a table join would be as follows:

tract,date,mean_temp
014501,2010-06-01,73
014402,2010-06-01,75
014703,2010-06-01,75
014100,2010-06-01,76
014502,2010-06-01,75
014403,2010-06-01,75
014300,2010-06-01,71
014200,2010-06-01,72
013610,2010-06-01,68

Temperature data metadata would be as follows:

"String","Date","Integer"

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

To perform a table join, perform the following steps:

  1. Copy the code from the first information box calls into a text file and save this as temperature.csv.

    Tip

    The CSVT format is a metadata file that accompanies a CSV file of the same name. It defines column data types.

  2. Copy the code from the second information box into a text file and save this as temperature.csvt. Otherwise, QGIS will not know what type of data is contained in each column.

    Tip

    Data for all the chapters will be found under the data directory for each chapter. You can use the included data under c1/data/original with the file names given earlier. Besides selecting the browse menu, you can also just drag the file into the Layers panel from an open operating system window. You can find examples of data output during exercises under the output directory of each chapter's data directory. This is also the directory given in the instructions as the destination directory for your output. You will probably want to create a new directory for your output and save your data there so as to not overwrite the included reference data.

  3. Navigate to Layer | Add Layer | Add Vector Layer | Browse to, and select temperature.csv.

    Tip

    CSV data can also be added through Layer | Add Layer | Add Delimited Text. This is especially useful to plot coordinates in a CSV, as you'll see later.

  4. Download the Tract boundary data:

    1. Visit http://www.census.gov/geo/maps-data/data/tiger-line.html.

    2. Click on the tab for the year you wish to find.

    3. Download the web interface.

    4. This will take us to http://www.census.gov/cgi-bin/geo/shapefiles2014/main.

    5. Navigate to Layer Type | Census Tracts and click on the submit button. Now, select Delaware from the Census Tract (2010) dropdown. Click on Submit again. Now select All counties in one state-based file from the dropdown displayed on this page and finally click on Download.

    6. Unzip the downloaded folder.

  5. Navigate to Layer | Add Layer | Add Vector Layer | Browse to, and select the tl_2010_10_tract10.shp file in the unzipped directory.

  6. Right-click on tl_2010_10_tract10 in the Layer panel, and then navigate to Properties | Joins. Click on the button with the green plus sign (+) to add a join.

  7. Select temperature as the Join layer option, tract as the Join field option, TRACTCE10 as the Target field option, and click on OK on this and the properties dialog:

To verify that the join completed, open the attribute table of the target layer (such as the geospatial reference, in this case, tl_2010_10) and sort by the new temperature_mean_temp field. Notice that the fields and values from the join layer are now included in the target layer.

  1. Select the target layer, tl_2010_10_tract10, from the Layers panel.

  2. Navigate to Layer | Open attribute table.

  3. Click on the temperature_mean_temp column header to sort tracts by this column. You may have to click twice to toggle the sort order from ascending to descending.

Geocode

If our data is expressed as addresses, intersections, or other well-known places, we can geocode it (that is, match it with coordinates) with a local or remote geocoder configured for our particular set of fields, such as the standard fields in an address.

In this example, we will geocode it using the remote geocoder provided by Google. Perform the following steps:

  1. Install the MMQGIS plugin.

  2. If you don't already have some address data to work with, you can make up a delimited file that contains some standard address fields, such as street, city, state, and county (ZIP code is not used by this plugin). The data that I'm using comes from New Castle County, Delaware's GIS site (http://gis.nccde.org/gis_viewer/).

  3. Whether you've downloaded your address data or made up your own, make sure to create a header row. Otherwise, MMQGIS fails to geocode.

    The following is an example of MMQGIS-friendly address data:

    id,address,city,state,zip,country
    1801300170,44 W CLEVELAND AV,NEWARK,DE,19711,USA
    1801400004,85 N COLLEGE AV,NEWARK,DE,19711,USA
    1802600068,501 ACADEMY ST,NEWARK,DE,19716,USA
  4. Open the MMQGIS geocode dialog by navigating to MMQGIS | Geocode | Geocode CSV with Google/OpenStreetMap.

  5. Once you've matched your fields to the address input fields available, you have the option of choosing Google Maps or OpenStreetMap. Google Maps usually have a much higher rate of success, while OpenStreetMap has the value of not having a daily limit on the number of addresses you can geocode. At this time, the OSM geocoder produces such poor results as to not be useful.

  6. You'll want to manually select or input a filesystem path for a notfound.csv file for the final input. The default file location can be problematic.

  7. Once your geocode is complete, you'll see how well the geocode address text matched with our geocoder reference. You may wish to alter addresses in the notfound.csv file and attempt to geocode these again.

Orthorectify

Finally, if our data is an image or grid (raster), we can match up locations in the image with known locations in a reference map. The registration of these pairs and subsequent transformation of the grid is called orthorectification or sometimes by the more generic term, georeferencing (even though that applies to a wider range of operations).

  1. Add a basemap, to be used for reference:

    1. Add the OpenLayers plugin. Navigate to Plugins | Manage | Install Plugins; select OpenLayers Plugin and click on Install.

    2. Navigate to Web | OpenLayers plugin, and select the basemap of your choice. MapQuest-OSM is a good option.

  2. Obtain map image:

    1. I have downloaded a high-resolution image (c1/data/original/4622009.jpg) from David Rumsey Map Collection, MapRank Search (http://rumsey.mapranksearch.com/), which is an excellent source for historical map images of the United States.

    2. Search by a location, filtering by time, scale, and other attributes. You can find the image we use by searching for Newark, Delaware.

    3. Once you find your map, navigate to it. Then, find Export in the upper right-hand corner, and export an extra high-resolution image.

    4. Unzip the downloaded folder.

  3. Orthorectify/georeference the image with the following steps:

    1. Install and enable the Georeferencer GDAL plugin.

    2. Navigate to Raster | Georeferencer | Georeferencer.

    3. Pan and zoom the reference basemap in the canvas on a location that you recognize in the map image.

    4. Pan and zoom on the map image.

    5. Select Add Control Point if it is not already selected.

    6. Click on the location in the map image that you recognized in the third step.

    7. Click the Pencil icon to choose control point from Map Canvas.

    8. Click on the location in the reference basemap.

    9. Click on OK.

    10. Add three of these control points, as shown in the following screenshot:

    11. Start georeferencing by clicking on the Play button.

    12. Enter the transformation settings information, as shown in the following screenshot:

  4. Now, start georeferencing by clicking on the Play button again.

Once your image has been georeferenced, you should see it align with the other data on your map. You can alter the layer transparency under Layer properties | Transparency:

The spatial reference manipulation – making the coordinates line up

QGIS will sometimes do an On-the-Fly (OTF) projection of all the data added to the canvas on the project CRS (defined under Project | Project Properties | CRS). You will want to disable OTF projection in the projects you intend to produce for web applications, as all layers should have their own spatial reference independently defined and transformed or projected in the same CRS, if needed.

Setting CRS

When geospatial data is received with no metadata on what the spatial reference system describes its coordinates, it is necessary to assign a system. This can be by right-clicking on the layer in Layers Panel | Save as and selecting the new CRS.

Transformation and projection

At other times, data is received with a different CRS than in the case of the other data used in the project. When CRSs differ, care should be taken to see whether to alter the CRS of the new nonconforming data or of the existing data. Of course we want to choose a system that supports our needs for accuracy or extent; at other times when we already have a suitable basemap, we will want operational layers to conform to the basemap's system. When a suitable basemap is already available to be consumed by our web application, we can often use the system of the basemap for the project. All major third-party basemap providers use Web Mercator, which is now known as EPSG:3857.

You can project data from geographic to projected coordinates or from one projection to another. This can be done in the same way as you would define a projection: by right-clicking on a layer in Layers Panel | Save as and selecting the new CRS. An appropriate transformation will generally be applied by default.

There are some features in CRS Selector that you should be aware of. By selecting from Recently used coordinate reference systems, you can often easily match up a new CRS with those existing in the workspace. You also have the option to search through the available systems by entering the Filter input. You will see the PROJ.4 WKT representation of the selected CRS at the bottom of the dialog.

     
About the Author
  • Ben Mearns

    Ben Mearns was previously the Lead Geospatial Information Consultant at the University of Delaware, where he was the senior GIS professional charged with leading, developing, and advising on GIS solutions. He developed the GeoServer-based campus mapping system. He has held GIS and data positions at the University of Pennsylvania Cartographic Modeling Lab, Princeton University Department of Sociology, and Macalester College Department of Geography.

    Browse publications by this author
Latest Reviews (6 reviews total)
Even if the book has got many interesting topicsand it is well written, it is often difficult to obtain data from external sources (e.g. internet website and so on) because of the lack of information given by the book authors, hence several times it is impossible to go on
Seems like a good source to learn from.
a well-written book with clear content and relevant examples
QGIS Blueprints
Unlock this book and the full library FREE for 7 days
Start now