NumPy Cookbook

5 (1 reviews total)
By Ivan Idris
  • Instant online access to over 8,000+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Winding Along with IPython

About this book

Today's world of science and technology is all about speed and flexibility. When it comes to scientific computing, NumPy is on the top of the list. NumPy will give you both speed and high productivity.

"NumPy Cookbook" will teach you all about NumPy, a leading scientific computing library. NumPy replaces a lot of the functionality of Matlab and Mathematica, but in contrast to those products, it is free and open source.

"Numpy Cookbook" will teach you to write readable, efficient, and fast code that is as close to the language of Mathematics as much as possible with the cutting edge open source NumPy software library.

You will learn about installing and using NumPy and related concepts. At the end of the book, we will explore related scientific computing projects.

This book will give you a solid foundation in NumPy arrays and universal functions. You will also learn about plotting with Matplotlib and the related SciPy project through examples.

"NumPy Cookbook" will help you to be productive with NumPy and write clean and fast code.

Publication date:
October 2012
Publisher
Packt
Pages
226
ISBN
9781849518925

 

Chapter 1. Winding Along with IPython

In this chapter, we will cover the following topics:

  • Installing IPython

  • Using IPython as a shell

  • Reading manual pages

  • Installing Matplotlib

  • Running a web notebook

  • Exporting a web notebook

  • Importing a web notebook

  • Configuring a notebook server

  • Exploring the SymPy profile

 

Introduction


IPython, which is available at http://ipython.org/, is a free, open source project available for Linux, Unix, Mac OS X, and Windows. The IPython authors only request that you cite IPython in any scientific work where IPython was used. It provides the following components, among others:

  • Interactive Python shells (terminal-based and Qt application)

  • A web notebook (available in IPython 0.12 and later) with support for rich media and plotting

  • IPython is compatible with Python versions 2.5, 2.6, 2.7, 3.1, and 3.2

You can try IPython in cloud without installing it on your system, by going to the following URL: http://www.pythonanywhere.com/try-ipython/. There is a slight delay compared to locally installed software; so this is not as good as the real thing. However, most of the features available in the IPython interactive shell seem to be available. They also have a Vi (m) editor, which if you like vi, is of course great. You can save and edit files from your IPython sessions. The author of this book doesn't care much about other editors, such as the one that starts with E and ends with macs. This should, however, not be a problem.

 

Installing IPython


IPython can be installed in various ways depending on your operating system. For the terminal-based shell, there is a dependency on readline. The web notebook requires tornado and zmq.

In addition to installing IPython, we will install setuptools, which gives you the easy_install command. The easy_install command is the default, standard package manager for Python. pip can be installed once you have easy_install available. The pip command is similar to easy_install, and adds options such as uninstalling.

How to do it...

This section describes how IPython can be installed on Windows, Mac OS X, and Linux. It also describes how to install IPython and its dependencies with easy_install and pip, or from source.

  • Installing IPython and setup tools on Windows: A binary Windows installer for Python 2 or Python 3 is available on the IPython website. Also see http://ipython.org/ipython-doc/stable/install/install.html#windows.

    Install setuptools with an installer from http://pypi.python.org/pypi/setuptools#files. Then install pip; for instance:

    cd C:\Python27\scripts
    python .\easy_install-27-script.py pip
    
  • Installing IPython On Mac OS X: Install the Apple Developer Tools (Xcode) if necessary. Xcode can be found on the OSX DVDs that came with your Mac or App Store. Follow the easy_install/pip instructions, or the installing from source instructions provided later in this section.

  • Installing IPython On Linux: Because there are so many Linux distributions, this section will not be exhaustive.

    • On Debian, type the following command:

      su – aptitude install ipython python-setuptools
      
    • On Fedora, the magic command is as follows:

      su – yum install ipython python-setuptools-devel
      
    • The following command will install IPython on Gentoo:

      su – emerge ipython
      
    • For Ubuntu, the install command is as follows:

      sudo apt-get install ipython python-setuptools
      
  • Installing IPython with easy_install or pip: Install IPython and all the dependencies required for the recipes in this chapter with easy_install, using the following command:

    easy_install ipython pyzmq tornado readline
    

    Alternatively, you can first install pip with easy_install, by typing the following command in your terminal:

    easy_install pip
    

    After that, install IPython using pip, with the following command:

    sudo pip install ipython pyzmq tornado readline
    
  • Installing from source: If you want to use the bleeding edge development version, then installing from source is for you.

    1. Download the latest tarball from https://github.com/ipython/ipython/downloads.

    2. Unpack the source code from the archive:

      tar xzf ipython-<version>.tar.gz
      
    3. If you have Git installed, you can clone the Git repository instead:

      $ git clone https://github.com/ipython/ipython.git
      
    4. Go to the ipython directory:

      cd ipython
      
    5. Run the setup script. This may require you to run the command with sudo, as follows:

      sudo setup.py install
      

How it works...

We installed IPython using several methods. Most of these methods install the latest stable release, except when you install from source, which will install the development version.

 

Using IPython as a shell


Scientists and engineers are used to experimenting. IPython was created by scientists with experimentation in mind. The interactive environment that IPython provides is viewed by many as a direct answer to Matlab, Mathematica, and Maple and R.

Following is a list of features of the IPython shell:

  • Tab completion

  • History mechanism

  • Inline editing

  • Ability to call external Python scripts with %run

  • Access to system commands

  • The pylab switch

  • Access to Python debugger and profiler

How to do it...

This section describes how to use the IPython shell.

  • The pylab switch: The pylab switch automatically imports all the Scipy, NumPy, and Matplotlib packages. Without this switch, we would have to import these packages ourselves.

    All we need to do is enter the following instruction on the command line:

    $ ipython -pylab
    Type "copyright", "credits" or "license" for more information.
    
    IPython 0.12 -- An enhanced Interactive Python.
    ?         -> Introduction and overview of IPython's features.
    %quickref -> Quick reference.
    help      -> Python's own help system.
    object?   -> Details about 'object', use 'object??' for extra details.
    
    Welcome to pylab, a matplotlib-based Python environment [backend: MacOSX].
    For more information, type 'help(pylab)'.
    In [1]: quit()
    quit() or Ctrl 
    + D quits the IPython shell.
    
  • Saving a session: We might want to be able to go back to our experiments. In IPython, it is easy to save a session for later use, with the following command:

    In [1]: %logstart
    Activating auto-logging. Current session state plus future input saved.
    Filename       : ipython_log.py
    Mode           : rotate
    Output logging : False
    Raw input log  : False
    Timestamping   : False
    State          : active
    

    Logging can be switched off as follows:

    In [9]: %logoff
    Switching logging OFF
    
  • Executing system shell commands: Execute system shell commands in the default IPython profile by prefixing the command with the ! symbol. For instance, the following input will get the current date:

    In [1]: !date
    

    In fact, any line prefixed with ! is sent to the system shell. Also, we can store the command output, as shown here:

    In [2]: thedate = !date
    In [3]: thedate
    
  • Displaying history: We can show the history of commands with the %hist command () for example:

    In [1]: a = 2 + 2
    
    In [2]: a
    Out[2]: 4
    
    In [3]: %hist
    a = 2 + 2
    a
    %hist
    

    This is a common feature in Command Line Interface (CLI) environments. We can also search through the history with the -g switch

    In [5]: %hist -g a = 2
        1: a = 2 + 2
    

Note

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

How it works...

We saw a number of so called "magic functions" in action. These functions start with the % character. If the magic function is used on a line by itself, the % prefix is optional.

 

Reading manual pages


When we are in IPython's pylab mode, we can open manual pages for NumPy functions with the help command. It is not necessary to know the name of a function. We can type a few characters and then let tab completion do its work. Let's, for instance, browse the available information for the arange function.

How to do it...

We can browse the available information, in either of the following two ways:

  • Calling the help function: Call the help command. Type a few characters of the function and press the Tab key:

  • Querying with a question mark: Another option is to put a question mark behind the function name. You will then, of course, need to know the function name, but you don't have to type help:

    In [3]: arange?
    

How it works...

Tab completion is dependent on readline, so you need to make sure it is installed. The question mark gives you information from docstrings.

 

Installing Matplotlib


Matplotlib is a very useful plotting library, which we will need for the next recipe. It depends on NumPy, but in all likelihood you already have NumPy installed.

How to do it...

We will see how Matplotlib can be installed in Windows, Linux, and Mac, and also how to install it from source.

  • Installing Matplotlib on Windows: Install with the Enthought distribution (http://www.enthought.com/products/epd.php).

    It might be necessary to put the msvcp71.dll file in your C:\Windows\system32 directory. You can get it from http://www.dll-files.com/dllindex/dll-files.shtml?msvcp71.

  • Installing Matplotlib on Linux: Let's see how Matplotlib can be installed in the various distributions of Linux:

    • The install command on Debian and Ubuntu is as follows:

      sudo apt-get install python-matplotlib
      
    • The install command on Fedora/Redhat is as follows:

      su - yum install python-matplotlib
      
  • Installing from source: Download the latest source from the tar.gz release at Sourceforge (http://sourceforge.net/projects/matplotlib/files/) or from the Git repository using the following command:

    git clone git://github.com/matplotlib/matplotlib.git
    

    Once it has been downloaded, build and install as usual with the following command:

    cd matplotlib
    python setup.py install
    
  • Installing Matplotlib on Mac: Get the latest DMG file from http://sourceforge.net/projects/matplotlib/files/matplotlib/, and install it.

 

Running a web notebook


The newest release of IPython introduced a new exciting feature – the web notebook. A so called "notebook server" can serve notebooks over the web. We can now start a notebook server and have a web-based IPython environment. This environment has most of the features in the regular IPython environment. The new features include the following:

  • Displaying images and inline plots

  • Using HTML and Markdown in text cells

  • Importing and exporting of notebooks

Getting ready

Before we start, we should make sure that all the required software is installed. There is a dependency on tornado and zmq. See the Installing IPython recipe in this chapter for more information.

How to do it...

  • Running a notebook: We can start a notebook with the following code:

    $ ipython notebook
    
    [NotebookApp] Using existing profile dir: u'/Users/ivanidris/.ipython/profile_default'
    [NotebookApp] The IPython Notebook is running at: http://127.0.0.1:8888
    [NotebookApp] Use Control-C to stop this server and shut down all kernels.
    

    As you can see, we are using the default profile. A server started on the local machine at port 8888. We will learn how to configure these settings later on in this chapter. The notebook is opened in your default browser; this is configurable as well:

    IPython lists all the notebooks in the directory where you started the notebook. In this example no notebooks were found. The server can be stopped with Ctrl + C.

  • Running a notebook in the pylab mode: Run a web notebook in the pylab mode with the following command:

    $ ipython notebook --pylab
    

    This loads the Scipy, NumPy, and Matplotlib modules.

  • Running notebook with inline figures: We can display inline Matplotlib plots with the inline directive, using the following command:

    $ ipython notebook --pylab inline
    
  1. Create a notebook: Click on the New Notebook button to create a new notebook:

  2. Create an array: Create an array with the arange function. Type the command in the following screenshot, and press Enter:

    Next, enter the following command and press Enter. You will see the output as shown in Out [2] in the following screenshot:

  3. Plot the sinc function: Apply the sinc function to the array and plot the result, as shown in the following screenshot:

How it works...

The inline option lets you display inline Matplotlib plots. When combined with the pylab mode, you don't need to import the NumPy, SciPy, and Matplotlib packages.

See also

The Installing IPython recipe.

 

Exporting a web notebook


Sometimes you will want to exchange notebooks with friends or colleagues. The web notebook provides several methods to export your data.

How to do it...

A web notebook can be exported using the following options:

  • The Print option: The Print button doesn't actually print the notebook, but allows you to export the notebook as PDF or HTML document.

  • Downloading the notebook: Download your notebook to a location chosen by you, using the Download button. We can specify whether we want to download the notebook as .py file, which is just a normal Python program, or in the JSON format as a .ipynb file. The notebook we created in the previous recipe looks like the following, after exporting:

    {
     "metadata": {
      "name": "Untitled1"
     }, 
     "nbformat": 2, 
     "worksheets": [
      {
        "cells": [
        {
          "cell_type": "code", 
          "collapsed": false, 
          "input": [
            "plot(sinc(a))"
          ], 
          "language": "python", 
          "outputs": [
          {
            "output_type": "pyout", 
            "prompt_number": 3, 
            "text": [
              "[&lt;matplotlib.lines.Line2D at 0x103d9c690&gt;]"
            ]
          }, 
          {
            "output_type": "display_data", 
            "png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD9CAYAAABZVQdHAAAABHNCSVQICAgIf...
              mgkAAAAASUVORK5CYII=\n"
          }
          ], 
          "prompt_number": 3
        }
        ]
      }
      ]
    }

    Note

    Some of the text has been omitted for brevity. This file is not intended for editing or reading even, but it is pretty readable if you ignore the image representation part. For more information about JSON please see https://en.wikipedia.org/wiki/JSON.

  • Saving the notebook: Save the notebook using the Save button. This will automatically export a notebook in the native JSON .ipynb format. The file will be stored in the directory where you started IPython initially.

 

Importing a web notebook


Python scripts can be imported as a web notebook. Obviously, we can also import previously exported notebooks.

How to do it...

The following steps show how a python script can be imported as a web notebook:

  1. Import a python script by dragging it from Explorer or Finder into the notebook page. The following screenshot is an example of what we see after dragging the vectorsum.py from NumPy Beginner's Guide into the notebook page:

  2. Click the Upload button to import the program. IPython does a decent job of importing the code. Unfortunately, as shown in the following screenshot, the code is all placed in one cell. At least that is how it worked at the time of writing:

  3. Tag the script for multiple cells.

    In order to split the code into multiple cells we need to use special tags. These tags are in fact Python comments, but they look a bit like XML tags. The code has to start with the following tag:

    # <nbformat>2</nbformat>

    This indicates the format of the notebook. Each new code cell is indicated with the following tag:

    # <codecell>

    The following is the tagged code:

    # <nbformat>2</nbformat>
    #!/usr/bin/env/python
    
    from datetime import datetime
    import numpy
    """
     Chapter 1 of NumPy Beginners Guide.
     This program demonstrates vector addition the Python way.
     Run from the command line as follows
         
      python vectorsum.py n
     
     where n is an integer that specifies the size of the vectors.
    
     The first vector to be added contains the squares of 0 up to n. 
     The second vector contains the cubes of 0 up to n.
     The program prints the last 2 elements of the sum and the elapsed time.
    """
    
    def numpysum(n):
       a = numpy.arange(n) ** 2
       b = numpy.arange(n) ** 3
       c = a + b
    
       return c
    
    def pythonsum(n):
       a = range(n)
       b = range(n)
       c = []
    
       for i in range(len(a)):
           a[i] = i ** 2
           b[i] = i ** 3
           c.append(a[i] + b[i])
    
       return c
       
    # <codecell>
    size = int(50)
    
    # <codecell>
    start = datetime.now()
    c = pythonsum(size)
    delta = datetime.now() - start
    print "The last 2 elements of the sum", c[-2:]
    print "PythonSum elapsed time in microseconds", delta.microseconds
    
    # <codecell>
    start = datetime.now()
    c = numpysum(size)
    delta = datetime.now() - start
    print "The last 2 elements of the sum", c[-2:]
    print "NumPySum elapsed time in microseconds", delta.microseconds

    The code is split into several cells according to the tags, as shown in the following screenshot:

 

Configuring a notebook server


A public notebook server needs to be secure. You should set a password and use a SSL certificate to connect to it. We need the certificate to provide secure communication over https (for more information see https://en.wikipedia.org/wiki/Transport_Layer_Security).

How to do it...

The following steps describe how to configure a secure notebook server:

  1. Generate a password: We can generate a password from IPython. Start a new IPython session, and type in the following commands:

    In [1]: from IPython.lib import passwd
    
    In [2]: passwd()
    Enter password: 
    Verify password: 
    Out[2]: 'sha1:0e422dfccef2:84cfbcbb3ef95872fb8e23be3999c123f862d856'
    

    At the second input line you will be prompted for a password. You need to remember this password. A long string is generated. Copy this string because we will need it later on.

  2. Create a SSL certificate: To create a SSL certificate, you will need to have the openssl command in your path.

    Setting up the openssl command is not rocket science, but can be tricky. Unfortunately, it is outside the scope of this book. On the bright side there are plenty of tutorials available online to help you further.

    Execute the following command to create a certificate with the name mycert.pem:

    $ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem
    Generating a 1024 bit RSA private key
    ......++++++
    ........................++++++
    writing new private key to 'mycert.pem'
    -----
    You are about to be asked to enter information that will be incorporated
    into your certificate request.
    What you are about to enter is what is called a Distinguished Name or a DN.
    There are quite a few fields but you can leave some blank
    For some fields there will be a default value,
    If you enter '.', the field will be left blank.
    -----
    Country Name (2 letter code) [AU]:
    State or Province Name (full name) [Some-State]:
    Locality Name (eg, city) []:
    Organization Name (eg, company) [Internet Widgits Pty Ltd]:
    Organizational Unit Name (eg, section) []:
    Common Name (eg, YOUR name) []:
    Email Address []:
    

    The openssl utility prompts you to fill in some fields. For more information, check the relevant man page (short for manual page).

  3. Create a server profile: Create a special profile for the server using the following command:

    ipython profile create nbserver
    
  4. Edit the profile configuration file: Edit the configuration file. In this example, it can be found in Edit in ~/.ipython/profile_nbserver/ipython_notebook_config.py.

    The configuration file is pretty large, so we will omit many of the lines in it. The lines that we need to change at minimum are:

    c.NotebookApp.certfile = u'/absolute/path/to/your/certificate'
    c.NotebookApp.password = u'sha1:b...your password'
    c.NotebookApp.port = 9999
    

    Notice that we are pointing to the SSL certificate we created. We set a password and changed the port to 9999.

  5. Start the server: Using the following command, start the server to check whether the changes worked.

    ipython notebook --profile=nbserver
    [NotebookApp] Using existing profile dir: u'/Users/ivanidris/.ipython/profile_nbserver'
    [NotebookApp] The IPython Notebook is running at: https://127.0.0.1:9999
    [NotebookApp] Use Control-C to stop this server and shut down all kernels.
    

    The server is running on port 9999, and you need to connect to it via https. If everything goes well, we should see a login page. Also, you would probably need to accept a security exception in your browser.

How it works...

We created a special profile for our public server. There are some sample profiles that are already present, such as the default profile. Creating a profile adds a profile_<profilename> folder to the .ipython directory with, among others, a configuration file. The profile can then be loaded with the --profile=<profile_name> command-line option. We can list the profiles with the following command:

ipython profile list

Available profiles in IPython:
    cluster
    math
    pysh
    python3

    The first request for a bundled profile will copy it
    into your IPython directory (/Users/ivanidris/.ipython),
    where you can customize it.

Available profiles in /Users/ivanidris/.ipython:
    default
    nbserver
    sh
 

Exploring the SymPy profile


IPython has a sample SymPy profile. SymPy is a Python symbolic, mathematics library. For instance, we can simplify algebraic expressions or differentiate, similar to Mathematica and Maple. SymPy is obviously a fun piece of software, but is not directly necessary for our journey through the NumPy landscape. Consider this as an optional bonus recipe. Like dessert, feel free to skip, although you might miss out on the sweetest piece of this chapter.

Getting ready

Install SymPy using either easy_install, or pip:

easy_install sympy
sudo pip install sympy

How to do it...

  1. Look at the configuration file, which can be found at ~/.ipython/profile_sympy/ipython_config.py. The contents are as follows:

    c = get_config()
    app = c.InteractiveShellApp
    
    # This can be used at any point in a config file to load a sub config
    # and merge it into the current one.
    load_subconfig('ipython_config.py', profile='default')
    
    lines = """
    from __future__ import division
    from sympy import *
    x, y, z, t = symbols('x y z t')
    k, m, n = symbols('k m n', integer=True)
    f, g, h = symbols('f g h', cls=Function)
    """
    
    # You have to make sure that attributes that are containers already
    # exist before using them.  Simple assigning a new list will override
    # all previous values.
    if hasattr(app, 'exec_lines'):
        app.exec_lines.append(lines)
    else:
        app.exec_lines = [lines]
    
    # Load the sympy_printing extension to enable nice printing of sympy expr's.
    if hasattr(app, 'extensions'):
        app.extensions.append('sympyprinting')
    else:
        app.extensions = ['sympyprinting']

    This code accomplishes the following:

    • Loading the default profile

    • Importing the SymPy packages

    • Defining symbols

  2. Start IPython with the SymPy profile using the following command:

    ipython --profile=sympy
    
  3. Expand an algebraic expression using the command shown in the following screenshot:

About the Author

  • Ivan Idris

    Ivan Idris has an MSc in Experimental Physics. His graduation thesis had a strong emphasis on Applied Computer Science. After graduating, he worked for several companies as a Java Developer, Data warehouse Developer, and QA Analyst. His main professional interests are Business Intelligence, Big Data, and Cloud Computing. Ivan Idris enjoys writing clean, testable code and interesting technical articles. Ivan Idris is the author of NumPy 1.5 Beginner's Guide and NumPy Cookbook by Packt Publishing. You can find more information and a blog with a few NumPy examples at ivanidris.net.

    Browse publications by this author

Latest Reviews

(1 reviews total)
Excellent
Book Title
Access this book, plus 8,000 other titles for FREE
Access now