Python Geospatial Analysis Cookbook

Before anything, we are going to assume that you already have a Linux/Ubuntu machine or a virtualbox instance running Linux/Ubuntu so you can follow these instructions.

Ubuntu 14.04 comes with Python 2.7.6 and Python 3.4 preinstalled; the other libraries are your responsibility as explained in the following sections.

Windows users need to download and install Python 2.7.x from the Python home page at https://www.python.org/downloads/windows/; please download the newest version of the 2.7.x series since this book is written with 2.7.X in mind. The installer includes a bundled version of pip, so make sure you install it!

Take a close look at the correct version to download, making sure that you get either the 32-bit or 64-bit download. You cannot mix and match the versions, so be careful and remember to install the correct version.

A great site for other kinds of Windows binaries can be found at http://www.lfd.uci.edu/~gohlke/pythonlibs/. Wheel files are the new norms of installations and can be executed from the command line as follows:

python pip install libraryName.whl

Python 3 would be awesome to use, and for many Python GIS libraries, it is ready for show time. Unfortunately, not all GIS libraries jive with Python 3 (pyproj) as one would love to see at the time of writing this. If you want, feel free to go for Python 3.x and give it a go. A great webpage to check the compatibility of a library can be found at https://caniusepython3.com/.

To install virtualenv, you need to have a running installation of Python and pip. The pip package manager manages and installs Python packages, making our lives easier. Throughout this book, if we need to install a package, pip will be our tool of choice for this job. The official installation instructions for pip can be found at https://pip.pypa.io/en/latest/installing.html. To install pip from the command line, we first need to install easy_install. Let's try it out from the Terminal:

$ sudo apt-get install python-setuptools python-pip

With this one line, you have both pip and easy_install installed.

The command sudo means to run an execution as a super user. If this fails, you will need to get theez_setup.py file, which is available at https://bootstrap.pypa.io/ez_setup.py. After downloading the file, you can run it from the command line:

$ python ez_setup.py

Now pip should be up and running and you can execute commands to complete the installations of virtualenv and virtualenvwrapper. The virtualenvwrapper creates shortcuts that are faster ways to create or delete your virtual environments. You can test it as follows:

$ pip install virtualenv

The steps to install your Python virtualenv and virtualenvwrapper packages are as follows:

Step one shows how pip installs the virtualenv package into your system-wide Python installation. Step two shows how the virtualenvwrapper helper package is installed with easy_install because the virtualenvwrapper.sh file is not created using the pip installer. This will help us create, enter, and generally, work or switch between Python virtual environments with ease. Step three assigns the WORKON_HOME variable to a directory where we want to have all of our virtual environments. Then, we'll create a new directory to hold all the virtual environments. In step four, the command source is used to execute the shell script to set up the virtualenvwrapper package. In step five, we see how to actually create a new virtualenv called pygeoan_cb in our /home/mdiener/venvs directory. This final step automatically starts our virtualenv session.

Once the virtualenv session starts, we can now see the name of virtualenv in brackets like this:

(pygeoan_cb)mdiener@mdiener-VirtualBox:~$

To exit virtualenv, simply type the following code:

$ deactivate

Now, your command line should be back to normal as shown here:

mdiener@mdiener-VirtualBox:~$

To reactivate virtualenv, simply type:

$ workon pygeoan_cb

Inside the /venvs folder, you will find specific individual virtual environments for each project in the form of a subfolder. The virtualenvwrapper package will always create a new folder for each new project you create. You can, therefore, easily delete a folder and it will remove your virtual environment.

To quickly print a list all of the installed libraries to a file, we'll use the pip command:

$ pip freeze > requirements.txt

This will create a text file called requirements.txt in the current folder. The text file contains a list of all the installed Python packages inside the Python virtual environment currently running.

To create a new virtualenv from a requirements file, use the following command:

$ pip install -r /path/to/requirements.txt

For those of you who are just starting out with geospatial Python development, it should be noted that you should keep your project-specific code at another location outside your Python virtual environment folder. For example, I always have each project-related code contained in a separate folder called 01_projects, which is my main folder. The path to my projects folder is /home/mdiener/01_projects, and the structure of two of my projects is as follows:

All virtual environments are located under /home/mdiener/venvs/. Usually, I give them the same name as a project to keep things organized, as follows:

Fire up your virtual environment, if it is not already running, using the following standard start command:

$ workon pygeoan_cb

Your prompt should now look like this:

(pygeoan_cb)mdiener@mdiener-VirtualBox:~$

Now, we need to install some Python tools for development that allow us to install NumPy, so run this command:

$ sudo apt-get install -y python-dev

You are now ready to move on and install pyproj and NumPy inside your running virtual environment.

Simply fire up virtualenv and we will use the pip installer to do all the heavy lifting as follows:

No errors, I hope. You have now successfully installed NumPy and pyproj.

This easy installation works using the standard pip installation method. No tricks or special commands are needed. You need to simply execute the pip install <library_name> command and you are off to the races.

To prepare for installation, it is necessary to install some global packages, such as libgeos_c, as these are required by Shapely. NumPy is also a requirement that we have already met and is also used by Shapely.

Install the requirements of matplotlib from the command line like this:

$ sudo apt-get install freetype* libpng-dev libjpeg8-dev

These are the dependencies of matplotlib, which can be seen on a Ubuntu 14.04 machine.

Follow these instructions:

Another test to see if all has gone well is to simply enter the Python console and try to import the packages, and if no errors occur, your console should show an empty Python cursor. The output should look like what is shown in the following code:

(pygeoan_cb)mdiener@mdiener-VirtualBox:~/venv$ python
Python 2.7.3 (default,  Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type “help”,  “copyright”, “credits”, or  “license” for more information.
>>> import shapely
>>> import matplotlib
>>> import descartes
>>>

# type exit() to return
>>> exit()

If any errors occur, Python usually provides some good clues as to where the problem is located and there is always Stack Overflow. For example, have a look at http://stackoverflow.com/questions/19742406/could-not-find-library-geos-c-or-load-any-of-its-variants/23057508#2305750823057508.

Here, the order in which you install the packages is very important. The descartes package depends on matplotlib, and matplotlib depends on NumPy plus freetype and libpng. This narrows you down to installing NumPy first, then matplotlib and its dependencies, and finally, descartes.

The installation itself is simple with pip and should be quick and painless. The tricky parts occur if libgeos_c is not installed properly, and you might need to install the libgeos-dev library.

Enter your virtual environment using the following command:

$ workon pygeoan_cb

Your prompt should now look like this:

(pygeoan_cb)mdiener@mdiener-VirtualBox:~$

The three installations are as follows:

To test your installation of pyshp, use the import shapefile type. The output should look like what is shown in the following output:

(pygeoan_cb)mdiener@mdiener-VirtualBox:~/venv$ python
Python 2.7.3 (default,  Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type “help”,  “copyright”, “credits”, or  “license” for more information.
>> import shapefile
>> import geojson
>> import pandas

As seen in the other modules, we've used the standard installation pip package to execute installations. There are no other dependencies to worry about, making for fast progress.

The dependency jungle we looked at earlier is back and we need three more universal installations to our Ubuntu system using apt-get install as follows:

$ sudo apt-get install libblas-dev liblapack-dev gfortran

Three dependencies are used for the SciPy installation. PySAL depends on SciPy so make sure to install SciPy first. Only IPython does not need any extra installations.

Start up your Python virtual environment with the following code:

mdiener@mdiener-VirtualBox:~$ workon pygeoan_cb
(pygeoan_cb)mdiener@mdiener-VirtualBox:~$

Let's look at these steps:

SciPy and PySAL libraries are both geared to help accomplish various spatial analysis duties. The choice of tool is based on the task at hand, so make sure that you check which library offers what function at the command prompt as follows:

>>> from scipy import spatial
>>> help(spatial)

The output should look like what is shown in the following screenshot:

GDAL isn't known to be the friendliest beast to install on Windows, Linux, or OSX. There are many dependencies and even more ways to install them. The descriptions are not all very straightforward. Keep in mind that this description is just one way of doing things and will not always work on all machines, so please refer to the online instructions for the latest and best ways to get your system up and running.

To start with, we will install some dependencies globally on our machine. After the dependencies have been installed, we will go into the global installation of GDAL for Python in our global site packages.

To globally install GDAL into our Python site packages, we will proceed with the following steps:

Windows 7 plus users should use the OSGeo4W windows installer (https://trac.osgeo.org/osgeo4w/).Find the following section on the web page and download your Windows version in 32-bit or 64-bit. Follow the graphical installer instructions and the GDAL installation will then be complete.

The GDAL installation encompasses both the raster (GDAL) and vector (OGR) tools in one. Within the GDAL install are five modules that can be separately imported into your project depending on your needs:

>>> from osgeo import gdal
>>> from osgeo import ogr
>>> from osgeo import osr
>>> from osgeo import gdal_array
>>> from osgeo import gdalconst
>>> python
>>> import osgeo
>>> help(osgeo)

To see what packages are included with your Python GDAL installation, we use the Python built-in help function to list what the OSGeo module has to offer. This is what you should see:

NAME
    osgeo - # __init__ for osgeo package.
FILE
    /usr/lib/python2.7/dist-packages/osgeo/__init__.py
MODULE DOCS
    http://docs.python.org/library/osgeo
PACKAGE CONTENTS
    _gdal
    _gdal_array
    _gdalconst
    _ogr
    _osr
    gdal
    gdal_array
    gdalconst
    gdalnumeric
    ogr
    osr
DATA
    __version__ = '1.10.0'
    version_info = sys.version_info(major=2, minor=7, micro=3, releaseleve...
VERSION
    1.10.0
(END)

At the time of writing this, the GDAL version is now bumped up to 2.0, and in developer land, this is old even before it gets printed. Beware that the GDAL 2.0 has compatibility issues and for this book, version 1.x.x is recommended.

The http://www.gdal.org homepage is always the best place for reference regarding any information about it. The OSGEO includes GDAL as a supported project, and you can find more information on it at http://www.osgeo.org.

To use GeoDjango, we will need to have a spatial database installed, and in our case, we will be using PostgreSQL with the PostGIS extension. GeoDjango also supports Oracle, Spatialite, and MySQL. The dependencies of PostGIS include GDAL, GEOS, PROJ.4, LibXML2, and JSON-C.

Start up your Python virtual environment as follows:

mdiener@mdiener-VirtualBox:~$ workon pygeoan_cb
(pygeoan_cb)mdiener@mdiener-VirtualBox:~$

Follow these steps. These are taken from the PostgreSQL homepage for Ubuntu Linux:

Windows users should be directed to the PostgreSQL Windows (http://www.postgresql.org/download/windows/) binaries provided by EnterpriseDB (http://www.enterprisedb.com/products-services-training/pgdownload#windows). Download the correct version and follow the installer instructions. PostGIS is also included in the list of extensions that you can directly install using the installer.

Installations using the apt-get Ubuntu installer and the Windows installers are simple enough in order to have PostgreSQL, PostGIS, and Django up and running. However, the inner workings of the installers are beyond the scope of this book.

To summarize all the installed libraries, take a look at this table: