NumPy Beginner's Guide - Second Edition

By Ivan Idris
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. NumPy Quick Start

About this book

NumPy is an extension to, and the fundamental package for scientific computing with Python. In today's world of science and technology, it is all about speed and flexibility. When it comes to scientific computing, NumPy is on the top of the list.


NumPy Beginner's Guide will teach you about NumPy, a leading scientific computing library. NumPy replaces a lot of the functionality of Matlab and Mathematica, but in contrast to those products, is free and open source.

Write readable, efficient, and fast code, which is as close to the language of mathematics as is currently possible with the cutting edge open source NumPy software library. Learn all the ins and outs of NumPy that requires you to know basic Python only. Save thousands of dollars on expensive software, while keeping all the flexibility and power of your favourite programming language.You will learn about installing and using NumPy and related concepts. At the end of the book we will explore some related scientific computing projects. This book will give you a solid foundation in NumPy arrays and universal functions. Through examples, you will also learn about plotting with Matplotlib and the related SciPy project. NumPy Beginner's Guide will help you be productive with NumPy and have you writing clean and fast code in no time at all.

Publication date:
April 2013
Publisher
Packt
Pages
310
ISBN
9781782166085

 

Chapter 1. NumPy Quick Start

Let's get started. We will install NumPy and related software on different operating systems and have a look at some simple code that uses NumPy. The IPython interactive shell is introduced briefly. As mentioned in the Preface, SciPy is closely related to NumPy, so you will see the SciPy name appearing here and there. At the end of this chapter, you will find pointers on how to find additional information online if you get stuck or are uncertain about the best way to solve problems.

In this chapter, we shall:

  • Install Python, SciPy, Matplotlib, IPython, and NumPy on Windows, Linux, and Macintosh

  • Write simple NumPy code

  • Get to know IPython

  • Browse online documentation and resources

 

Python


NumPy is based on Python, so it is required to have Python installed. On some operating systems, Python is already installed. However, you need to check whether the Python version corresponds with the NumPy version you want to install. There are many implementations of Python, including commercial implementations and distribution. In this book we will focus on the standard CPython implementation, which is guaranteed to be compatible with NumPy.

 

Time for action – installing Python on different operating systems


NumPy has binary installers for Windows, various Linux distributions, and Mac OS X. There is also a source distribution, if you prefer that. You need to have Python 2.4.x or above installed on your system. We will go through the various steps required to install Python on the following operating systems:

  1. Debian and Ubuntu: Python might already be installed on Debian and Ubuntu but the development headers are usually not. On Debian and Ubuntu install python and python-dev with the following commands:

    sudo apt-get install python
    sudo apt-get install python-dev
    
  2. Windows: The Windows Python installer can be found at www.python.org/download. On this website, we can also find installers for Mac OS X and source tarballs for Linux, Unix, and Mac OS X.

  3. Mac: Python comes pre-installed on Mac OS X. We can also get Python through MacPorts, Fink, or similar projects. We can install, for instance, the Python 2.7 port by running the following command:

    sudo port install python27
    

    LAPACK does not need to be present but, if it is, NumPy will detect it and use it during the installation phase. It is recommended to install LAPACK for serious numerical analysis as it has useful numerical linear algebra functionality.

What just happened?

We installed Python on Debian, Ubuntu, Windows, and the Mac.

 

Windows


Installing NumPy on Windows is straightforward. You only need to download an installer, and a wizard will guide you through the installation steps.

 

Time for action – installing NumPy, Matplotlib, SciPy, and IPython on Windows


Installing NumPy on Windows is necessary but, fortunately, a straightforward task that we will cover in detail. It is recommended to install Matplotlib, SciPy, and IPython. However, this is not required to enjoy this book. The actions we will take are as follows:

  1. Download a NumPy installer for Windows from the SourceForge website http://sourceforge.net/projects/numpy/files/.

    Choose the appropriate version. In this example, we chose numpy-1.7.0-win32-superpack-python2.7.exe.

  2. Open the EXE installer by double clicking on it.

  3. Now, we can see a description of NumPy and its features as shown in the previous screenshot. Click on the Next button.

  4. If you have Python installed, it should automatically be detected. If it is not detected, maybe your path settings are wrong. At the end of this chapter, resources are listed in case you have problems installing NumPy.

  5. In this example, Python 2.7 was found. Click on the Next button if Python is found; otherwise, click on the Cancel button and install Python (NumPy cannot be installed without Python). Click on the Next button. This is the point of no return. Well, kind of, but it is best to make sure that you are installing to the proper directory and so on and so forth. Now the real installation starts. This may take a while.

  6. Install SciPy and Matplotlib with the Enthought distribution http://www.enthought.com/products/epd.php. It might be necessary to put the msvcp71.dll file in your C:\Windows\system32 directory. You can get it from http://www.dll-files.com/dllindex/dll-files.shtml?msvcp71. A Windows IPython installer is available on the IPython website (see http://ipython.scipy.org/Wiki/IpythonOnWindows).

What just happened?

We installed NumPy, SciPy, Matplotlib, and IPython on Windows.

 

Linux


Installing NumPy and related recommended software on Linux depends on the distribution you have. We will discuss how you would install NumPy from the command line, although, you could probably use graphical installers; it depends on your distribution (distro). The commands to install Matplotlib, SciPy, and IPython are the same – only the package names are different. Installing Matplotlib, SciPy, and IPython is recommended, but optional.

 

Time for action – installing NumPy, Matplotlib, SciPy, and IPython on Linux


Most Linux distributions have NumPy packages. We will go through the necessary steps for some of the popular Linux distros:

  1. Run the following instructions from the command line for installing NumPy and Red Hat:

    yum install python-numpy
    
  2. To install NumPy on Mandriva, run the following command-line instruction:

    urpmi python-numpy
    
  3. To install NumPy on Gentoo run the following command-line instruction:

    sudo emerge numpy
    
  4. To install NumPy on Debian or Ubuntu, we need to type the following :

    sudo apt-get install python-numpy
    

The following table gives an overview of the Linux distributions and corresponding package names for NumPy, SciPy, Matplotlib, and IPython.

Linux distribution

NumPy

SciPy

Matplotlib

IPython

Arch Linux

python-numpy

python-scipy

python-matplotlib

ipython

Debian

python-numpy

python-scipy

python-matplotlib

ipython

Fedora

numpy

python-scipy

python-matplotlib

ipython

Gentoo

dev-python/numpy

scipy

matplotlib

ipython

OpenSUSE

python-numpy, python-numpy-devel

python-scipy

python-matplotlib

ipython

Slackware

numpy

scipy

matplotlib

ipython

What just happened?

We installed NumPy, SciPy, Matplotlib, and IPython on various Linux distributions.

 

Mac OS X


You can install NumPy, Matplotlib, and SciPy on the Mac with a graphical installer or from the command line with a port manager such as MacPorts or Fink, depending on your preference.

 

Time for action – installing NumPy, Matplotlib, and SciPy on Mac OS X


We will install NumPy with a GUI installer using the following steps:

  1. We can get a NumPy installer from the SourceForge website http://sourceforge.net/projects/numpy/files/. Similar files exist for Matplotlib and SciPy. Just change numpy in the previous URL to scipy or matplotlib. IPython didn't have a GUI installer at the time of writing. Download the appropriate DMG file as shown in the following screenshot, usually the latest one is the best:

  2. Open the DMG file as shown in the following screenshot (in this example, numpy-1.7.0-py2.7-python.org-macosx10.6.dmg):

    • Double-click on the icon of the opened box, the one having a subscript that ends with .mpkg. We will be presented with the welcome screen of the installer.

    • Click on the Continue button to go to the Read Me screen, where we will be presented with a short description of NumPy as shown in the following screenshot:

    • Click on the Continue button to the License the screen.

  3. Read the license, click on the Continue button and then on the Accept button, when prompted to accept the license. Continue through the next screens and click on the Finish button at the end.

What just happened?

We installed NumPy on Mac OS X with a GUI installer. The steps to install SciPy and Matplotlib are similar and can be performed using the URLs mentioned in the first step.

 

Time for action – installing NumPy, SciPy, Matplotlib, and IPython with MacPorts or Fink


Alternatively, we can install NumPy, SciPy, Matplotlib, and IPython through the MacPorts route or with Fink. The following installation steps shown install all these packages. We only need NumPy for all the tutorials in this book, so please omit the packages you are not interested in.

  1. For installing with MacPorts, type the following command:

    sudo port install py-numpy py-scipy py-matplotlib py-ipython
    
  2. Fink also has packages for NumPy: scipy-core-py24, scipy-core-py25, and scipy-core-py26. The SciPy packages are: scipy-py24, scipy-py25, and scipy-py26. We can install NumPy and the other recommended packages we will be using in this book for Python 2.6 with the following command:

    fink install scipy-core-py26 scipy-py26 matplotlib-py26
    

What just happened?

We installed NumPy and other recommended software on Mac OS X with MacPorts and Fink.

 

Building from source


We can retrieve the source code for NumPy with git as follows:

git clone git://github.com/numpy/numpy.git numpy

Install /usr/local with the following command:

python setup.py build
sudo python setup.py install --prefix=/usr/local

To build, we need a C compiler such as GCC and the Python header files in the python-dev or python-devel package.

 

Arrays


After going through the installation of NumPy, it's time to have a look at NumPy arrays. NumPy arrays are more efficient than Python lists, when it comes to numerical operations. NumPy code requires less explicit loops than equivalent Python code.

 

Time for action – adding vectors


Imagine that we want to add two vectors called a and b. Vector is used here in the mathematical sense meaning a one-dimensional array. We will learn in Chapter 5, Working with Matrices and ufuncs, about specialized NumPy arrays which represent matrices. The vector a holds the squares of integers 0 to n, for instance, if n is equal to 3, then a is equal to 0, 1, or 4. The vector b holds the cubes of integers 0 to n, so if n is equal to 3, then the vector b is equal to 0, 1, or 8. How would you do that using plain Python? After we come up with a solution, we will compare it with the NumPy equivalent.

  1. The following function solves the vector addition problem using pure Python without NumPy:

    def pythonsum(n):
       a = range(n)
       b = range(n)
       c = []
    
       for i in range(len(a)):
           a[i] = i ** 2
           b[i] = i ** 3
           c.append(a[i] + b[i])
    
       return c
  2. The following is a function that achieves the same with NumPy:

    def numpysum(n):
      a = numpy.arange(n) ** 2
      b = numpy.arange(n) ** 3
      c = a + b
      return c

Notice that numpysum() does not need a for loop. Also, we used the arange function from NumPy that creates a NumPy array for us with integers 0 to n. The arange function was imported; that is why it is prefixed with numpy.

Now comes the fun part. Remember that it is mentioned in the Preface that NumPy is faster when it comes to array operations. How much faster is Numpy, though? The following program will show us by measuring the elapsed time in microseconds, for the numpysum and pythonsum functions. It also prints the last two elements of the vector sum. Let's check that we get the same answers by using Python and NumPy:

#!/usr/bin/env/python

import sys
from datetime import datetime
import numpy as np

"""
 Chapter 1 of NumPy Beginners Guide.
 This program demonstrates vector addition the Python way.
 Run from the command line as follows
     
  python vectorsum.py n
 
 where n is an integer that specifies the size of the vectors.

 The first vector to be added contains the squares of 0 up to n. 
 The second vector contains the cubes of 0 up to n.
 The program prints the last 2 elements of the sum and the elapsed time.
"""

def numpysum(n):
   a = np.arange(n) ** 2
   b = np.arange(n) ** 3
   c = a + b

   return c

def pythonsum(n):
   a = range(n)
   b = range(n)
   c = []

   for i in range(len(a)):
       a[i] = i ** 2
       b[i] = i ** 3
       c.append(a[i] + b[i])

   return c
   
size = int(sys.argv[1])

start = datetime.now()
c = pythonsum(size)
delta = datetime.now() - start
print "The last 2 elements of the sum", c[-2:]
print "PythonSum elapsed time in microseconds", delta.microseconds

start = datetime.now()
c = numpysum(size)
delta = datetime.now() - start
print "The last 2 elements of the sum", c[-2:]
print "NumPySum elapsed time in microseconds", delta.microseconds

The output of the program for 1000, 2000, and 3000 vector elements is as follows:

$ python vectorsum.py 1000
The last 2 elements of the sum [995007996, 998001000]
PythonSum elapsed time in microseconds 707
The last 2 elements of the sum [995007996 998001000]
NumPySum elapsed time in microseconds 171
$ python vectorsum.py 2000
The last 2 elements of the sum [7980015996, 7992002000]
PythonSum elapsed time in microseconds 1420
The last 2 elements of the sum [7980015996 7992002000]
NumPySum elapsed time in microseconds 168
$ python vectorsum.py 4000
The last 2 elements of the sum [63920031996, 63968004000]
PythonSum elapsed time in microseconds 2829
The last 2 elements of the sum [63920031996 63968004000]
NumPySum elapsed time in microseconds 274

Tip

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

What just happened?

Clearly, NumPy is much faster than the equivalent normal Python code. One thing is certain; we get the same results whether we are using NumPy or not. However, the result that is printed differs in representation. Notice that the result from the numpysum function does not have any commas. How come? Obviously we are not dealing with a Python list but with a NumPy array. It was mentioned in the Preface that NumPy arrays are specialized data structures for numerical data. We will learn more about NumPy arrays in the next chapter.

Pop quiz Functioning of the arange function

Q1. What does arange(5) do?

  1. Creates a Python list of 5 elements with values 1 to 5.

  2. Creates a Python list of 5 elements with values 0 to 4.

  3. Creates a NumPy array with values 1 to 5.

  4. Creates a NumPy array with values 0 to 4.

  5. None of the above.

Have a go hero – continue the analysis

The program we used here to compare the speed of NumPy and regular Python is not very scientific. We should at least repeat each measurement a couple of times. It would be nice to be able to calculate some statistics such as average times, and so on. Also, you might want to show plots of the measurements to friends and colleagues.

Tip

Hints to help can be found in the online documentation and resources listed at the end of this chapter. NumPy has, by the way, statistical functions that can calculate averages for you. I recommend using Matplotlib to produce plots. Chapter 9, Plotting with Matplotlib, gives a quick overview of Matplotlib.

 

IPython—an interactive shell


Scientists and engineers are used to experimenting. IPython was created by scientists with experimentation in mind. The interactive environment that IPython provides is viewed by many as a direct answer to Matlab, Mathematica, and Maple. You can find more information, including installation instructions, at: http://ipython.org/.

IPython is free, open source, and available for Linux, Unix, Mac OS X, and Windows. The IPython authors only request that you cite IPython in scientific work where IPython was used. Here is the list of basic IPython features:

  • Tab completion

  • History mechanism

  • Inline editing

  • Ability to call external Python scripts with %run

  • Access to system commands

  • Pylab switch

  • Access to Python debugger and profiler

The Pylab switch imports all the Scipy, NumPy, and Matplotlib packages. Without this switch, we would have to import every package we need, ourselves.

All we need to do is enter the following instruction on the command line:

$ ipython --pylab
Python 2.7.2 (default, Jun 20 2012, 16:23:33) 
Type "copyright", "credits" or "license" for more information.

IPython 0.14.dev -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

Welcome to pylab, a matplotlib-based Python environment [backend: MacOSX].
For more information, type 'help(pylab)'.
In [1]: quit()

The quit() function or Ctrl + D quits the IPython shell. We might want to be able to go back to our experiments. In IPython, it is easy to save a session for later:

In [1]: %logstart
Activating auto-logging. Current session state plus future input saved.
Filename       : ipython_log.py
Mode           : rotate
Output logging : False
Raw input log  : False
Timestamping   : False
State          : active

Let's say we have the vector addition program that we made in the current directory. We can run the script as follows:

In [1]: ls
README        vectorsum.py
In [2]: %run -i vectorsum.py 1000

As you probably remember, 1000 specifies the number of elements in a vector. The -d switch of %run starts an ipdb debugger and on typing c, the script is started. n steps through the code. Typing quit at the ipdb prompt exits the debugger.

In [2]: %run -d vectorsum.py 1000
*** Blank or comment
*** Blank or comment
Breakpoint 1 at: /Users/…/vectorsum.py:3
><string>(1)<module>()
ipdb> c
> /Users/…/vectorsum.py(3)<module>()
      2
1---> 3 import sys
      4 from datetime import datetime
ipdb> n
>
/Users/…/vectorsum.py(4)<module>()
1     3 import sys
----> 4 from datetime import datetime
      5 import numpy
ipdb> n
> /Users/…/vectorsum.py(5)<module>()
      4 from datetime import datetime
----> 5 import numpy
      6 
ipdb> quit

We can also profile our script by passing the -p option to %run.

In [4]: %run -p vectorsum.py 1000
         1058 function calls (1054 primitive calls) in 0.002 CPU seconds
   Ordered by: internal time
ncallstottimepercallcumtimepercallfilename:lineno(function)
1 0.001    0.001   0.001    0.001 vectorsum.py:28(pythonsum)
1 0.001    0.001   0.002    0.002 {execfile}
1000 0.000    0.0000.0000.000 {method 'append' of 'list' objects}
1 0.000    0.000    0.002    0.002 vectorsum.py:3(<module>)
1 0.000    0.0000.0000.000 vectorsum.py:21(numpysum)
3    0.000    0.0000.0000.000 {range}
1    0.000    0.0000.0000.000 arrayprint.py:175(_array2string)
3/1    0.000    0.0000.0000.000 arrayprint.py:246(array2string)
2    0.000    0.0000.0000.000 {method 'reduce' of 'numpy.ufunc' objects}
4    0.000    0.0000.0000.000 {built-in method now}
2    0.000    0.0000.0000.000 arrayprint.py:486(_formatInteger)
2    0.000    0.0000.0000.000 {numpy.core.multiarray.arange}
1    0.000    0.0000.0000.000 arrayprint.py:320(_formatArray)
3/1    0.000    0.0000.0000.000 numeric.py:1390(array_str)
1    0.000    0.0000.0000.000 numeric.py:216(asarray)
2    0.000    0.0000.0000.000 arrayprint.py:312(_extendLine)
1    0.000    0.0000.0000.000 fromnumeric.py:1043(ravel)
2    0.000    0.0000.0000.000 arrayprint.py:208(<lambda>)
1    0.000    0.000    0.002    0.002<string>:1(<module>)
11    0.000    0.0000.0000.000 {len}
2    0.000    0.0000.0000.000 {isinstance}
1    0.000    0.0000.0000.000 {reduce}
1    0.000    0.0000.0000.000 {method 'ravel' of 'numpy.ndarray' objects}
4    0.000    0.0000.0000.000 {method 'rstrip' of 'str' objects}
3    0.000    0.0000.0000.000 {issubclass}
2    0.000    0.0000.0000.000 {method 'item' of 'numpy.ndarray' objects}
1    0.000    0.0000.0000.000 {max}
1    0.000    0.0000.0000.000 {method 'disable' of '_lsprof.Profiler' objects}

This gives us a bit more insight into the workings of our program. In addition, we can now identify performance bottlenecks. The %hist command shows the commands history.

In [2]: a=2+2
In [3]: a
Out[3]: 4
In [4]: %hist
1: _ip.magic("hist ")
2: a=2+2
3: a

I hope you agree that IPython is a really useful tool!

 

Online resources and help


When we are in IPython's pylab mode , we can open manual pages for NumPy functions with the help command. It is not necessary to know the name of a function. We can type a few characters and then let tab completion do its work. Let's, for instance, browse the available information for the arange function.

In [2]: help ar<Tab>
In [2]: help arange

Another option is to put a question mark behind the function name:

In [3]: arange?

The main documentation website for NumPy and SciPy is at http://docs.scipy.org/doc/. Through this webpage, we can browse the NumPy reference at http://docs.scipy.org/doc/numpy/reference/ and the user guide as well as several tutorials.

NumPy has a wiki with lots of documentation at http://docs.scipy.org/numpy/Front%20Page/.

The NumPy and SciPy forum can be found at http://ask.scipy.org/en.

The popular Stack Overflow software development forum has hundreds of questions tagged numpy. To view them, go to http://stackoverflow.com/questions/tagged/numpy.

If you are really stuck with a problem or you want to be kept informed of NumPy development, you can subscribe to the NumPy discussion mailing list. The e-mail address is . The number of e-mails per day is not too high and there is almost no spam to speak of. Most importantly, developers actively involved with NumPy also answer questions asked on the discussion group. The complete list can be found at http://www.scipy.org/Mailing_Lists.

For IRC users, there is an IRC channel on irc.freenode.net. The channel is called #scipy, but you can also ask NumPy questions since SciPy users also have knowledge of NumPy, as SciPy is based on NumPy. There are at least 50 members on the SciPy channel at all times.

 

Summary


In this chapter, we installed NumPy and other recommended software that we will be using in some tutorials. We got a vector addition program working and convinced ourselves that NumPy has superior performance. We were introduced to the IPython interactive shell. In addition, we explored the available NumPy documentation and online resources.

In the next chapter, we will take a look under the hood and explore some fundamental concepts including arrays and data types.

About the Author

  • Ivan Idris

    Ivan Idris has an MSc in experimental physics. His graduation thesis had a strong emphasis on applied computer science. After graduating, he worked for several companies as a Java developer, data warehouse developer, and QA analyst. His main professional interests are business intelligence, big data, and cloud computing. Ivan Idris enjoys writing clean, testable code and interesting technical articles. Ivan Idris is the author of NumPy 1.5. Beginner's Guide and NumPy Cookbook by Packt Publishing.

    Browse publications by this author
Book Title
Access this book, plus 7,500 other titles for FREE
Access now