Learning about Geospatial Analysis with Python

Geospatial technology is currently impacting our world since it is changing our knowledge of human history. In this book, we will step through the history of geospatial analysis, which predates computers and even paper maps. Then, we will examine why you might want to learn about and use a programming language as a geospatial analyst as opposed to just using geographic information system (GIS) applications. This will help us understand the importance of making geospatial analysis as accessible as possible to as many people as possible.

In this chapter, we will be covering the following topics:

• Geospatial analysis and our world
• Dr. Sarah Parcak and archaeology
• Geographic information systems
• Remote sensing concepts
• Elevation data
• Computer-aided drafting
• Geospatial analysis and computer programming
• The importance of geospatial analysis...

Technical requirements

This book assumes that you have some basic knowledge of the Python programming language, basic computer literacy, and at least an awareness of geospatial analysis. This chapter provides a foundation for geospatial analysis, which is needed to attack any subject in the areas of remote sensing and GIS, including the material in all the other chapters of this book.

The examples in this book are based on Python 3.4.3, which you can download here: https://www.python.org/downloads/release/python-343/.

Geospatial analysis and our world

In the 1880s, British explorers began applying scientific rigor to excavating ancient cultural sites. The field of archaeology is a frustrating, low, costly, and often dangerous endeavor requiring patience and a good bit of luck. The Earth is remarkably good at keeping secrets and erasing the story of human endeavors. Changing rivers, floods, volcanoes, dust storms, hurricanes, earthquakes, fires, and other events swallow entire cities into the surrounding landscape, and we lose them to the flow of time.

Our knowledge of human history is based on glimpses into ancient cultures through archaeological excavation and the study of sites we have been lucky enough to stumble across through educated guesses or trial and error. There used to be no success in archaeology unless a team excavated a site, found something, and correctly identified it. Predictions...

History of geospatial analysis

Geospatial analysis can be traced back to as far as 15,000 years ago, to the Lascaux cave in southwestern France. In this cave, Paleolithic artists painted commonly hunted animals and what many experts believe are astronomical star maps for either religious ceremonies or potentially even migration patterns of prey. Though crude, these paintings demonstrate an ancient example of humans creating abstract models of the world around them and correlating spatial-temporal features to find relationships. The following photograph shows one of the paintings, with an overlay illustrating the star maps:

Over the centuries, the art of cartography and the science of land surveying have developed, but it wasn't until the 1800s that significant advances in geographic analysis emerged. Deadly cholera outbreaks in Europe between 1830 and 1860 led geographers...

GIS

Computer mapping evolved with the computer itself in the 1960s. However, the origin of the term GIS began with the Canadian Department of Forestry and Rural Development. Dr. Roger Tomlinson headed a team of 40 developers in an agreement with IBM to build the Canada Geographic Information System (CGIS). The CGIS tracked the natural resources of Canada and allowed the profiling of those features for further analysis. The CGIS stored each type of land cover as a different layer.

It also stored data in a Canadian-specific coordinate system, suitable for the entire country, which was devised for optimal area calculations. While the technology that was used was primitive by today's standards, the system had phenomenal capability at that time. The CGIS included software features that seem quite modern:

• Map projection switching
• The rubber sheeting of scanned images
• Map scale...

Remote sensing

Remote sensing is where you collect information about an object without making physical contact with that object. In the context of geospatial analysis, that object is usually the Earth. Remote sensing also includes processing the collected information. The potential of geographic information systems is limited only by the available geographic data. The cost of land surveying, even using a modern GPS to populate a GIS, has always been resource-intensive.

The advent of remote sensing not only dramatically reduced the cost of geospatial analysis but took the field in entirely new directions. In addition to powerful reference data for GIS systems, remote sensing has made the automated and semi-automated generation of GIS data possible by extracting features from images and geographic data. The eccentric French photographer, Gaspard-Félix Tournachon, also known...

Elevation data

Remote sensing data can measure the Earth in two dimensions. But we can also use remote sensing to measure the Earth in three dimensions using digital elevation data, which we include in a Digital Elevation Model. A Digital Elevation Model (DEM) is a three-dimensional representation of a planet's terrain. In the context of this book, this planet is the Earth. The history of digital elevation models is far less complicated than remotely-sensed imagery but no less significant. Before computers, representations of elevation data were limited to topographic maps created through traditional land surveys. The technology existed to create three-dimensional models from stereoscopic images or physical models from materials such as clay or wood, but these approaches were not widely used for geography.

The concept of digital elevation models came about in 1986 when the...

Computer-aided drafting

Computer-aided drafting (CAD) is worth mentioning, though it does not directly relate to geospatial analysis. The history of CAD system development parallels and intertwines with the history of geospatial analysis. CAD is an engineering tool used to model two- and three-dimensional objects, usually for engineering and manufacturing. The primary difference between a geospatial model and CAD model is that a geospatial model is referenced to the Earth, whereas a CAD model can possibly exist in abstract space.

For example, a three-dimensional blueprint of a building in a CAD system would not have latitude or longitude, but in a GIS, the same building model would have a location on the Earth. However, over the years, CAD systems have taken on many features of GIS systems and are commonly used for smaller GIS projects. Likewise, many GIS programs can import CAD...

Geospatial analysis and computer programming

Modern geospatial analysis can be conducted with the click of a button in any of the easy-to-use commercial or open source geospatial packages. So, why would you want to use a programming language to learn this field? The most important reasons are as follows:

• You want complete control of the underlying algorithms, data, and execution
• You want to automate specific, repetitive analysis tasks with minimal overhead from a large, multipurpose geospatial framework
• You want to create a program that's easy to share
• You want to learn geospatial analysis beyond pushing buttons in software

The geospatial industry is gradually moving away from the traditional workflow, in which teams of analysts use expensive desktop software to produce geospatial products. Geospatial analysis is being pushed toward automated processes that reside in the...

The importance of geospatial analysis

Geospatial analysis helps people make better decisions. It doesn't make the decision for you, but it can answer critical questions that are at the heart of the choice to be made and often cannot be answered any other way. Until recently, geospatial technology and data were tools available only to governments and well-funded researchers. However, in the last decade, data has become much more widely available and software has become much more accessible to anyone.

In addition to freely available government satellite imagery, many local governments now conduct aerial photo surveys and make the data available online. The ubiquitous Google Earth provides a cross-platform spinning globe view of the Earth with satellite and aerial data, streets, points of interest, photographs, and much more. Google Earth users can create custom Keyhole Markup...

GIS concepts

In order to begin geospatial analysis, we need to understand some key underlying concepts that are unique to the field. The list isn't long, but nearly every aspect of analysis traces back to one of these ideas.

Thematic maps

As its name suggests, a thematic map portrays a specific theme. A general reference map visually represents features as they relate geographically to navigation or planning. A thematic map goes beyond location to provide the geographic context for information around a central idea. Usually, a thematic map is designed for a targeted audience to answer specific questions. The value of thematic maps lies in what they do not show. A thematic map will use minimal geographic features to avoid...

Remote sensing concepts

Most of the GIS concepts we've described also apply to raster data. However, raster data has some unique properties as well. Earlier in this chapter, when we went over the history of remote sensing, the focus was on Earth imaging from aerial platforms. It is important to note that raster data can come in many forms, including ground-based radar, laser range finders, and other specialized devices to detect gases, radiation, and other forms of energy in a geographic context.

For the purpose of this book, we will focus on remote sensing platforms that capture large amounts of Earth data. These sources included Earth imaging systems, certain types of elevation data, and some weather systems, where applicable.

Images as data

...

Common vector GIS concepts

In this section, we will discuss the different types of GIS processes that are commonly used in geospatial analysis. This list is not exhaustive; however, it provides you with the essential operations that all other operations are based on. If you understand these operations, you will quickly understand much more complex processes as they are either derivatives or combinations of these processes.

Data structures

GIS vector data uses coordinates consisting of, at a minimum, an x horizontal value and a y vertical value to represent a location on the Earth. In many cases, a point may also contain a z value. Other ancillary values are possible, including measurements or timestamps.

These coordinates...

Common raster data concepts

As we mentioned earlier, remotely sensed raster data is a matrix of numbers. Remote sensing contains thousands of operations that can be performed on data. This field changes on almost a daily basis as new satellites are put into space and computer power increases.

Despite its decade-long history, we haven't even scratched the surface of the knowledge that this field can provide to the human race. Once again, similar to the common GIS processes, this minimal list of operations allows you to evaluate any technique that's used in remote sensing.

Band math

Band math is multidimensional array mathematics. In array math, arrays are treated as single units, which are added, subtracted, multiplied...

Creating the simplest possible Python GIS

Now that we have a better understanding of geospatial analysis, the next step is to build a simple GIS known as SimpleGIS using Python. This small program will be a technically complete GIS with a geographic data model and the ability to render the data as a visual thematic map showing the population of different cities.

The data model will also be structured so that you can perform basic queries. Our SimpleGIS will contain the state of Colorado, three cities, and population counts for each city.

Most importantly, we will demonstrate the power and simplicity of Python programming by building this tiny system in pure Python. We will only use modules available in the standard Python distribution without downloading any third-party libraries.

...

Getting an overview of common data formats

As a geospatial analyst, you may frequently encounter the following general data types:

• Spreadsheets and comma-separated values (CSV files) or tab-separated values (TSV files)
• Geotagged photos
• Lightweight binary points, lines, and polygons
• Multi-gigabyte satellite or aerial images
• Elevation data such as grids, point clouds, or integer-based images
• XML files
• JSON files
• Databases (both servers and file databases)
• Web services
• Geodatabases

Each format contains its own challenges for access and processing. When you perform analysis on data, you usually have to do some form of preprocessing first. You might clip or subset a satellite image of a large area down to just your area of interest, or you might reduce the number of points in a collection to just the ones meeting certain criteria in your data model. A good example of this type...

Understanding data structures

Despite having dozens of formats, geospatial data has some common traits. Understanding these traits can help you approach and understand unfamiliar data formats by identifying the ingredients common to nearly all spatial data. The structure of a given data format is usually driven by its intended use.

Some data is optimized for efficient storage or compression, some is optimized for efficient access, some is designed to be lightweight and readable (web formats), while other data formats seek to contain as many different data types as possible.

Interestingly, some of the most popular formats today are also some of the simplest and even lack features found in more capable and sophisticated formats. Ease of use is extremely important to geospatial analysts because so much time is spent integrating data into geographic information systems, as well as...