Real-Time Data

A common saying among geospatial analysts is: A map is outdated as soon as it's created. This saying reflects the fact that the Earth and everything on it are constantly changing. For most of the history of geospatial analysis and through most of this book, geospatial products are relatively static. Raw datasets are typically updated anywhere from a few months to a few years. The age of geospatial data in a map is referred to as data currency.

Data currency has traditionally not been the primary focus because of the time and expense needed to collect data. Web mapping, wireless cellular modems, and low-cost GPS antennas have changed that focus. It is now logistically feasible and even quite affordable to monitor a rapidly changing object or system and broadcast those changes to millions of people online. This change is revolutionizing geospatial technology...

Technical requirements

This chapter requires the following things:

• Python 3.6 or higher
• RAM: Minimum 6 GB (Windows), 8 GB (macOS), recommended 8 GB
• Storage: Minimum 7200 RPM SATA with 20 GB of available space, and recommended SSD with 40 GB of available space
• Processor: Minimum Intel Core i3 2.5 GHz, and recommended Intel Core i5
• The MapQuest Developer API key, available here: https://developer.mapquest.com/plan_purchase/steps/business_edition/business_edition_free/register

Limitations of real-time data

The term real-time data typically means near-real-time. Some tracking devices capture real-time data and may update as often as several times a second. But the limitations of the infrastructure that broadcasts that data may constrain the output to every 10 seconds or longer. Weather radar is a perfect example. A Doppler Weather Radar (DWR) sweeps continuously but data is typically available online every five minutes. But given the contrast with traditional geospatial data updates, a refresh of a few minutes is real-time enough. Limitations can be summarized as follows:

• Network bandwidth limitations restricting data size
• Network latency limiting the data update frequency
• Availability of the data source due to restrictions such as battery life
• Lack of quality control due to data being instantly available to consumers
• Security vulnerabilities due to...

Using real-time data

Web mashups often use real-time data. Web mashups are amazing and have changed the way many different industries operate. But they are typically limited in that they usually just display pre-processed data on a map and give developers access to a JavaScript API. But what if you want to process the data in some way? What if you want to filter, change, and then send it to another system? To use real-time data for geospatial analysis, you need to be able to access it as point data or a georeferenced raster.

You can find out more about web map mashups here: https://www.esri.com/arcgis-blog/products/product/uncategorized/digital-map-mashups/.

As with examples in the previous chapters, the scripts are as simple as possible and designed to be read from start to finish without much mental looping. When functions are used they are listed first, followed by script variable...

Tracking vehicles

For our first real-time data source, we'll use the excellent NextBus API. NextBus (http://www.nextbus.com/) is a commercial service that tracks public transportation for municipalities including buses, trolleys, and trains. People riding these transit lines can then track the arrival time of the next bus.

What's even better is that, with the customer's permission, NextBus publishes tracking data through a REpresentational State Transfer (REST) API. Using URL API calls, developers can request information about a vehicle and receive an XML document about its location. This API is a straightforward way to begin using real-time data.

If you go to NextBus, you'll see a web interface as shown in the following screenshot, showing data for the city of Los Angeles, California metro system:

The system lets you select several parameters to learn the...

Storm chasing

So far, we have created a simpler version of what the NextBus website already does. But we have done it in a way that ultimately gives us complete control over the output. Now we want to use this control to go beyond what the NextBus Google Maps mashup does. We'll add another real-time data source that is very important to both travelers and bus-line operators: the weather.

Iowa State University's Mesonet program provides free and polished weather data for applications. We use this data to create a real-time weather map for our bus location map. We can use the Open Geospatial Consortium (OGC) Web Map Service (WMS) standard to request a single image over our area of interest. A WMS is an OGC standard for serving georeferenced map images through the web; they are generated by a map server through an HTTP request.

The Mesonet system provides an excellent web...

Reports from the field

In our final example in this chapter, we'll get off of the bus and out into the field. Modern smartphones, tablets, and laptops allow us to update a GIS and view those updates from everywhere. We'll use HTML, GeoJSON, the Leaflet JavaScript library, and a pure-Python library named Folium to create a client-server application that allows us to post geospatial information to a server and then create an interactive web map to view those data updates.

First, we need a web form that shows your current location and updates the server when you submit the form with comments about your location. You can find the form here: http://geospatialpython.github.io/Learn/fieldwork.html.

The following screenshot shows the form:

You can view the source of that form to see how it works. The mapping is done using the Leaflet library and posts GeoJSON to a unique URL...

Summary

Real-time data is an exciting way to do new types of geospatial analysis, only recently made possible by advances in several different technologies, including web mapping, GPS, and wireless communications. In this chapter, you learned how to access raw feeds for real-time location data, how to acquire a subset of real-time raster data, how to combine different types of real-time data into a custom map analysis product using only Python, and how to build client-server geospatial applications to update a GIS in real-time.

As with previous chapters, these examples contain building blocks that will let you build new types of application using Python that go far beyond the typical popular and ubiquitous JavaScript-based mashups.

In the next chapter, we will combine everything we've learned so far into a complete geospatial application that applies algorithms and concepts...