Chapter 2. The matplotlib Architecture

As software systems age, they tend to undergo a natural evolution through processes such as feature addition and debugging. The resultant codebase embodies the familiar tension between maintaining the old code and at the same time offering the end users an improved product. Architectures for long-term projects are not something that were originally carved in stone and adhered to monomaniacally ever since. Rather, they are living, adaptive concepts that guide the plans and activities of a project's contributors.

The matplotlib module arose out of such an environment, and it has continuous goals of refining and improving its architecture and updating its older bits to follow the best practices of and the latest advances in not only the project itself, but also the wider Python community over the years since its inception.

In this chapter, we will perform the following tasks:

• Review the original design goals of matplotlib and explore its evolution

• Examine...

The original design goals

As mentioned in Chapter 1, Getting Up to Speed, the creators of matplotlib were originally focused on building a GTK+ application for researchers and providing a command interface for the interactive plotting of data, not unlike that provided by MATLAB.

Both of these aims helped drive the development of improved abstractions for matplotlib. It was in this dual crucible that the top-level object of the rendered plots in matplotlib gained its rightful prominence—the Figure. These ideas led to various foundational objects in matplotlib, and the relationships between them ultimately provided the basis for the architecture of this library.

The current matplotlib architecture

The current matplotlib architecture revolves around the operations that are necessary for the users to create, render, and update the Figure objects. Figures can be displayed and interacted with via common user interface events such as the keyboard and mouse inputs. This layer of interaction with common user interface is called the backend layer. A Figure needs to be composed of multiple objects that should be individually modifiable, but it should be implemented in such a way that it has a positive and predictable impact on the other aspects of the Figure. This logical layer is responsible for the abstraction of each visual component that one sees in a Figure. Due to its highly visual nature, this layer was identified as the more general concept of creating visual art and is thus referred to as the artist layer. Lastly, the Figure needs to support programmatic interaction and provide the users with the ability to manipulate Figures with a syntax that...

The backend layer

Seasoned computer scientists, engineers, and software developers all know that one of the subtler and trickier problems that arise in our industry is naming. It sounds a bit silly and it is repeatedly the subject of jokes, but the difficulty remains—how do you speak or write explicitly on a subject whose very nature requires exquisite precision and yet has great ambiguity that arises in different contexts?

We have the same problem with the term backend. Here, as in so many other instances, the context is everything. Our context is matplotlib, a set of tools, and a framework where everything is done in support of the visualizing of data and their relationships. The term backend has to be viewed from this perspective to support the generation of plots. The matplotlib backend has nothing to do with other noteworthy backends such as databases, servers, messaging systems, or dispatchers of various sorts. The backend of matplotlib is an abstraction layer over various components...

The artist layer

The artist layer constitutes the bulk of what matplotlib actually does—the generation of the plots for the purpose of display, manipulation, and publication. Most work in the artist layer is performed by a number of classes, most of which are derived from the Artist base class.

The artist layer is concerned with things such as the lines, shapes, axes, text, and so on. These are the subclasses of the Artist class that define things such as the following:

• A canvas-artist coordinate transformation

• Visibility

• A clip box that defines the paintable area

• Labels

• A callback registry instance to handle user interaction events

The Artist subclasses can be classified into one of the following two groups:

• Primitives

• Containers

The following two sections provide more details about these groups.

Primitives

The matplotlib artist primitives are classes of graphical objects that are supposed to be painted on a figure's canvas. These include, but are not limited to, the following:

• Line2D

• Shape (patch) classes...

The scripting layer

While the backend layer focuses on providing a common interface to the toolkits and rendering the primitives and containers of the artist layer, the scripting layer is the user-facing interface that simplifies the task of working with other layers.

Programmers who integrate matplotlib with application servers will often find it more convenient to work directly with the backend and artist layers. However, for the scientists' daily use, data visualization, or exploratory interactions, pyplot—the scripting layer—is a better option. This is what we use in most of the IPython Notebooks in this book.

The pyplot interface is much less verbose; one can get insights into one's data in very few steps. Under the covers, pyplot uses module-level objects to track the state of the data so that the user does not have to create things like figures, axes, canvases, figure canvas managers, or preferred backends.

We will take a quick look at pyplot's internals later in this chapter (as well...

The supporting components of the matplotlib stack

In addition to the three major components of the matplotlib stack, there are supporting components. These include the following:

• Configuration support

• Utility modules and functions

• C extensions

• External libraries upon which matplotlib depends

We will touch on these in the coming chapters. They are related to the given topics at hand, but they do not impact the structure or nature of matplotlib's overall architecture.

Combining the details uncovered in the previous sections, the following diagram portrays a logical architecture for matplotlib that glosses over the finer details:

To make this more of a living reality, we will follow this high-level description with some more detailed examinations, which consist of the following:

• Learning about matplotlib modules and associated namespaces

• Creating a sample import graph

• Following the execution of the pyplot functions through the matplotlib stack

matplotlib modules

When discussing the architecture of software libraries, it is of great use to relate a conceptual overview to concrete software components. This not only increases the immediate knowledge with more definite context, but also provides a foundation for a quicker learning process during future explorations. Let's examine the modules in the matplotlib Python package.

Exploring the filesystem

Start by obtaining the IPython Notebook for this chapter, installing the dependencies, starting up the IPython server, and loading the notebook in your browser in the following way:

$ git clone https://github.com/masteringmatplotlib/architecture.git
$ cd architecture
$ make

Once the notebook is loaded, go ahead and run the initial setup commands:

In [1]: import matplotlib
        matplotlib.use('nbagg')
        %matplotlib inline

Now let's create two sets of imports, one for our dependencies and the other for modules that we've created specifically for this notebook:

In [2]: from glob import...

The execution flow

At the beginning of this chapter, we briefly sketched the flow of data from user creation to its display in a user interface. Having toured matplotlib's architecture, which included taking a side trip to the namespaces and dependency graphs, there is enough context to appreciate the flow of data through the code.

As we trace through our simple line example, remember that we used the pyplot interface. There are several other ways by which one may use matplotlib. For each of these ways, the code execution flow will be slightly different.

An overview of the script

As a refresher, here's our code from simple-line.py:

#! /usr/bin/env python3.4
import matplotlib.pyplot as plt

def main () -> None:
  plt.plot([1,2,3,4])
  plt.ylabel('some numbers')
  plt.savefig('simple-line.png')

if __name__ == '__main__':
  main()

At the script level, here's what we've got:

1. Operating system shell executes the script.

2. Python 3.4 is invoked, which then runs the script.

3. matplotlib is imported.

4. A main...

The matplotlib architecture as it relates to this book

The previous sections covered some heavy material, and it can be sometimes difficult to remember the big picture when examining the details under the proverbial microscope. As a preventative measure, we'd like to bring the discussion back to a macroscopic scale as it relates to knowledge acquisition.

The three layers that we've talked about in matplotlib's architecture are the backend, artist, and the scripting layers. As an intermediate user of matplotlib, you've very likely used all the three layers. However, you've most probably spent a lot of of time on the scripting layer with pyplot. This is where most users of matplotlib not only start, but usually stay. The introductory material in this book focuses on getting the users up to speed with the scripting layer so that they can be as effective as possible. Intermediate materials (such as the books for reference mentioned earlier in the chapter) also focus on this.

An in-depth usage...

Summary

In this chapter, we covered a lot of detailed material by starting with a high-level logical overview of matplotlib's structure. We then learned about the source code and examined how the matplotlib modules were laid out. We figured out the modules that were associated with different logical layers. Finally, we peered into the depths of matplotlib by tracing the function and method calls through the code. This was done by using a sample script in an interactive Python session.

One of the most commonly recommended practices for open source developers who want to improve is to read copious amounts of source code for the projects that you use the most and care about deeply. Similarly, as you develop your mastery of matplotlib, you will find yourself spending more and more time reading the source, exploring its depths, and ultimately making contributions to the project. This chapter is your first step in this direction.

The next step is learn more about the scripting layer and its best...