NumPy Essentials: Boost your scientific and analytic capabilities in no time at all by discovering how to build real-world applications with NumPy

Let's begin by taking a brief tour of the Scientific Python (SciPy) stack.

Figure 1: The SciPy stack, standard, and extended libraries

Fernando Perez, the primary author of IPython, said in his keynote at PyCon, Canada 2012:

This is precisely why the SciPy stack boasts such rich functionality. The evolution of most of the SciPy stack is motivated by teams of scientists and engineers trying to solve scientific and engineering problems in a general-purpose programming language. A one-line explanation of why NumPy matters so much is that it provides the core multidimensional array object that is necessary for most tasks in scientific computing. This is why it is at the root of the SciPy stack. NumPy provides an easy way to interface with legacy Fortran and C/C++ numerical code using time-tested scientific libraries, which we know have been working well for decades. Companies and labs across the world use Python to glue together legacy code that has been around for a long time. In short, this means that NumPy allows us to stand on the shoulders of giants; we do not have to reinvent the wheel. It is a dependency for every other SciPy package. The NumPy ndarray object, which is the subject of the next chapter, is essentially a Pythonic interface to data structures used by libraries written in Fortran, C, and, C++. In fact, the internal memory layouts used by NumPy ndarray objects implement C and Fortran layouts. This will be addressed in detail in upcoming chapters.

The next layer in the stack consists of SciPy, matplotlib, IPython (the interactive shell of Python; we will use it for the examples throughout the book, and details of its installation and usage will be provided in later sections), and SymPy modules. SciPy provides the bulk of the scientific and numerical functionality that a major part of the ecosystem relies on. Matplotlib is the de facto plotting and data visualization library in Python. IPython is an increasingly popular interactive environment for scientific computing in Python. In fact, the project has had such active development and enjoyed such popularity that it is no longer limited to Python and extends its features to other scientific languages, particularly R and Julia. This layer in the stack can be thought of as a bridge between the core array-oriented functionality of NumPy and the domain-specific abstractions provided by the higher layers of the stack. These domain-specific tools are commonly called SciKits-popular ones among them are scikit-image (image processing), scikit-learn (machine learning), statsmodels (statistics), pandas (advanced data analysis), and so on. Listing every scientific package in Python would be nearly impossible since the scientific Python community is very active, and there is always a lot of development happening for a large number of scientific problems. The best way to keep track of projects is to get involved in the community. It is immensely useful to join mailing lists, contribute to code, use the software for your daily computational needs, and report bugs. One of the goals of this book is to get you interested enough to actively involve yourself in the scientific Python community.

A fundamental question that beginners ask is. Why are arrays necessary for scientific computing at all? Surely, one can perform complex mathematical operations on any abstract data type, such as a list. The answer lies in the numerous properties of arrays that make them significantly more useful. In this section, let's go over a few of these properties to emphasize why something such as the NumPy ndarray object exists at all.

It is said that, if you stand at Times Square long enough, you will meet everyone in the world. By now, you must have been convinced that NumPy is the Times Square of SciPy. If you are writing scientific applications in Python, there is not much you can do without digging into NumPy. Figure 2 shows the scope of SciPy in scientific computing at varying levels of abstraction. The red arrow denotes the various low-level functions that are expected of scientific software, and the blue arrow denotes the different application domains that exploit these functions. Python, armed with the SciPy stack, is at the forefront of the languages that provide these capabilities.

A Google Scholar search for NumPy returns nearly 6,280 results. Some of these are papers and articles about NumPy and the SciPy stack itself, and many more are about NumPy's applications in a wide variety of research problems. Academics love Python, which is showcased by the increasing popularity of the SciPy stack as the primary language of scientific programming in countless universities and research labs all over the world. The experiences of many scientists and software professionals have been published on the Python website:

Figure 2: Python versus other languages

Now that the credibility of Python and NumPy has been established, let's get our hands dirty.

The default environment used for all Python code in this book will be IPython. Instructions on how to install IPython and other tools follow in the next section. Throughout the book, you will only have to enter input in either the command window or the IPython prompt. Unless otherwise specified, code will refer to Python code, and command will refer to bash or DOS commands.

All Python input code will be formatted in snippets like these:

 In [42]: print("Hello, World!")

In [42]: in the preceding snippet indicates that this is the 42^nd input to the IPython session. Similarly, all input to the command line will be formatted as follows:

 $ python hello_world.py 

On Windows systems, the same command will look something like this:

C:\Users\JohnDoe> python hello_world.py

For the sake of consistency, the $ sign will be used to denote the command-line prompt, regardless of OS. Prompts, such as C:\Users\JohnDoe>, will not appear in the book. While, conventionally, the $ sign indicates bash prompts on Unix systems, the same commands (without typing the actual dollar sign or any other character), can be used on Windows too. If, however, you are using Cygwin or Git Bash, you should be able to use Bash commands on Windows too.

Note that Git Bash is available by default if you install Git on Windows.

Let's take a look at the various requirements we need to set up before we proceed.

$ pip install numpy
$ easy_install numpy
$ enpkg numpy # for Canopy users
$ conda install numpy # for Anaconda users

 In [1]: import numpy as np 
 In [2]: np.test()

In this chapter, we introduced ourselves to the NumPy module. We took a look at how NumPy is a useful software tool to have for those of you who are working in scientific computing. We installed the software required to proceed through the rest of this book.

In next chapter, we will get to the powerful NumPy ndarray object, showing you how to use it efficiently.