In this chapter, we cover the following recipes:

• Installing Anaconda on Windows
• Installing Anaconda on macOS
• Installing Anaconda on Linux
• Checking the Anaconda installation
• Installing SciPy from a binary distribution on Windows
• Installing SciPy from a binary distribution on macOS
• Installing SciPy from source on Linux
• Installing optional packages with conda
• Installing packages with pip
• Setting up a virtual environment with conda
• Creating a virtual environment for development with conda
• Creating a conda environment with a different version of a package
• Using conda environments to run different versions of Python
• Creating Virtual environments with venv
• Running SciPy in a script
• Running SciPy in Jupyter
• Running SciPy in Spyder
• Running SciPy in PyCharm

In this chapter, we discuss the available options for setting up and running the SciPy stack and associated tools. We present solutions for all the major platforms and consider different scenarios. Readers are advised to browse through all installation options before deciding which option fits their workflow and computational needs. After reading this chapter, the reader will understand all different options for setting up a full-fledged environment in Python for computational and data science.

The recipes in this chapter assume the use of the following tools:

• The Command Prompt, also known as Terminal in Linux and macOS. Each operating system has a different way of accessing the default Command Prompt:
  • In Windows, open the search bar and type cmd.
  • In macOS, the Terminal app is in the Applications-Utilities folder.
  • In Linux, the Command Prompt may be called xterm or Terminal. In Ubuntu, it can also be started by pressing Ctrl + Alt + T.
• A text editor. Sublime Text is a popular multi-platform programmer's editor with many nice features, available at:
  https://www.sublimetext.com. Sublime Text is commercial software, but a trial version is available.
  Alternatives available for each operating system are as follows:
  • Windows: Notepad is pre-installed on Windows. A free Notepad alternative that adds nice features for free is Notepad++, which can be downloaded from https://notepad-plus-plus.org.
  • macOS: TextEdit is pre-installed and can be found in the Applications folder. An alternative is nano, a simple text editor that can be started from a Terminal window.
  • Linux: Usually ships with at least one of the following: gedit, nano, or vim, all of which can all be launched from a Terminal window.

If you decide to use the pre-assembled Anaconda distribution, you will also need to download it from the following site: https://www.continuum.io/downloads.

Choose the latest 64-bit Python 3 distribution, unless you have an older computer with a 32-bit architecture.

In this recipe, we will show you how to install Anaconda on a Windows system.

1. Double-click the downloaded .exe installer
2. Accept the software license
3. When prompted for the kind of installation you want, select Just Me, and then click Next
4. Accept the default installation folder
5. In the next option box, select both Add Anaconda to my PATH environment variable and Register Anaconda as my default Python 3.x
6. Click Install to finish the installation
7. Anaconda will be installed in the C:\Users\username\Anaconda3 folder
8. Optionally, proceed to the Checking the Anaconda installation

In this recipe, we will show you how to install Anaconda on a macOS system.

1. Double-click the downloaded installer file, which is a file with a .pkg extension
2. Click the Continue button to view ReadMe and accept the software license
3. When prompted, select the Install for me only option and click Continue
4. Review the installation options and click the Install button
5. Wait until the installation finishes, and then click the Close button to quit the installer
6. Anaconda will be installed in the anaconda subfolder of your home folder
7. Optionally, proceed to the Checking the Anaconda installation recipe

In this recipe, we will show you how to install Anaconda on a Linux system.

1. Open a Terminal window on the folder containing the installer and run the following command, replacing the version number, x.x.x, with the corresponding value for file you downloaded:

bash Anaconda3-x.x.x-Linux-x86_64.sh

2. Review the license agreement and accept it. Enter yes when prompted to continue the installation
3. When asked if you want to prepend the Anaconda3 installation location to your path, answer yes
4. Wait until the installer stops
5. Anaconda will be installed in the anaconda3 subfolder of your home folder.
6. Optionally, proceed to the Checking the Installation recipe

This recipe shows you how to do some basic checking. We will verify that the software can be started and that the correct version is being used.

1. Open a new Terminal window and run the following command:

python3

2. Verify the information displayed in the Terminal. It will look like the following:

Python 3.6.0 |Anaconda 4.3.1 (x86_64)| (default, Dec 23 2016, 13:19:00) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

3. Notice that, in the output, we can check that we are indeed running the Python 3 version distributed with Anaconda.

You have now successfully installed Anaconda on your computer.

Windows does not ship with any version of Python pre-installed, which actually makes things easier when we want to install our own version of Python.

On the other hand, the installation of a full SciPy stack in Windows is somewhat more complex, due to conflicts that exist between the Python distribution and certain Windows libraries. We indicate an installation route that has been tested several times, but some trial and error may be necessary due to changes in the distribution.

To make the instructions easier to follow, the installation procedure is broken down into two stages:

• Installing Python
• Install the SciPy stack

1. Go to https://www.python.org and download the Python 3 binary distribution for Windows
2. Once the download finishes, double-click on the installation file to start the setup
3. Check the box Add Python 3.x to PATH
4. Click the Install Now option
5. Select the Disable the path length limit option, if available, on the last installation screen
6. Close the installation screen

These steps will install Python in the folder:

C:\Users\username\Appdata\Local\Programs\Python\Python3x

To test the installation, start a Command Prompt window and enter the following command:

python3

If all is correct, the Python command-line interpreter will start and display information about the version of Python being run. For now, just exit the interpreter by entering, at the >>> Python prompt, the following statement:

quit()

Now, let's check if pip was correctly installed. Enter the following at the command line:

pip --version

This should print information about the currently installed version of pip3, including the location where packages will be installed. As long as no errors are reported, the installation is correct.

To install SciPy, we need to first download the versions of the library that have been built specifically for Windows. They can be found at the following site: http://www.lfd.uci.edu/~gohlke/pythonlibs/.

This page contains a long list of pre-compiled Python packages for Windows. Search the page for numpy-mkl and scipy and look for a package that matches your operating system and Python distribution. In my case, I found the following two files:

numpy-1.12/1+mkl-cp36-cp36m-win_amd64.whl
scipy-0.19.0-cp36-cpm36m-win_amd64.whl

Notice that the package names refer to version 3.6 and a 64-bit architecture. Make sure the versions you download match your Python 3 distributions. Open a command window on the directory where the files were saved and enter the following two commands, in the following order:

pip install numpy-1.12/1+mkl-cp36-cp36m-win_amd64.whl
pip install scipy-0.19.0-cp36-cpm36m-win_amd64.whl

After installing NumPy and SciPy, pip can be used to install the other packages directly by running the commands shown as follows:

pip install matplotlib
pip install ipython jupyter
pip install pandas sympy nose

Let's now test the installation. First, start Python 3 and execute the following statements at the >>> Python prompt:

import numpy
import scipy
import matplotlib
import pandas
import IPython
import sympy

If you can run all these commands and there are no errors, the installation of the packages is correct. Exit the Python shell by running the following statement:

quit()

Now, back in the command window, run the following command:

ipython

This will start IPython and display information about the installed version. For now, simply exit IPython by running the following at the prompt:

quit()

Finally, let's test the Jupyter Notebook. At the command line, run the following command:

jupyter notebook

If all is correct, this will start the Jupyter Notebook in your browser after a few seconds. This finishes the installation of Python and the SciPy stack in Windows.

macOS currently ships with version 2.7 of Python pre-installed. In this recipe, we will show you how to install Python 3 on the Mac without making changes to the original Python distribution. The easiest way to achieve this is to use Homebrew, a package manager for macOS.

The full installation instructions are broken down into the following stages:

• Installing the Xcode command-line tools
• Installing Homewbrew
• Installing Python 3
• Installing the SciPy stack

Xcode is the free development environment for macOS distributed by Apple. If you already have Xcode installed on your computer, you can skip this step. If you don't, open a Terminal window and run the following command:

xcode-select --install

If you get an error message, then the command-line tools are already installed and you can go to the next step. Otherwise, a window will pop up asking for confirmation. Press the Install button and, when prompted, accept the license agreement.

To check that the command-line tools were correctly installed, run the following command in the Terminal:

gcc -v

This command prints information about the gcc compiler present in your computer, which will be similar to the output shown as follows:

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.38)
Target: x86_64-apple-darwin16.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

If you get no error message, the command-line tools are properly installed.

Homebrew is a package manager for macOS that makes it easier to install and remove software packages without interfering with system software that ships with the computer. It installs package files to the /usr/local directory and makes no changes to system folders. Although it is possible to install Python on the Mac from the source, using Homebrew considerably simplifies the setup process.

To install Homebrew, open a Terminal window and run the following command. Please note that the whole command should be typed in a single Terminal line:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Follow the on-screen instructions and confirm that you want to install Homebrew. Enter the administrative password for your computer if prompted. On a personal computer, this is usually the same as your login password.

To check that Homebrew was successfully installed, run the following command:

brew -v

This command outputs information about the current Homebrew installation, which looks like the following example:

Homebrew 1.1.13
Homebrew/homebrew-core (git revision c80e; last commit 2017-04-26)

If you get a similar message and no errors, Homebrew is properly installed.

Once Homebrew is set up, install Python 3 by running the following command from a Terminal window:

brew install python3

The installation process will start and may take a few minutes. When it is finished, run the following from the command line:

python3

If the installation is correct, this will print information about the Python interpreter, shown as follows:

Python 3.x.x (default, Apr  4 2017, 09:40:21) 
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

You can check that you are indeed running the Python distribution that you installed by checking the version number, indicated by 3.x.x in the preceding sample. You can now exit Python by running the following command at the >>> Python prompt:

quit()

We are going to use the pip3 Python package manager to install the SciPy stack. To check that pip3 was correctly installed, run the following statement from the command line:

pip3 --version

This will print to the Terminal information about the currently installed version of pip, as shown in the following example:

pip 9.0.1 from /home/fmartins/python3/lib/python3.x/site-packages (python 3.x)

Verify that you are indeed running the version of pip3 associated with your installation of Python by checking the version number, indicated by 3.x in the preceding sample output. If no error message is issued, the setup was completed correctly, and you can proceed to install SciPy.

To install the SciPy stack, execute each of the following commands on a Terminal window:

pip3 install --user numpy scipy matplotlib
pip3 install --user ipython jupyter 
pip3 install --user pandas sympy nose

We now need to adjust the PATH variable in the .bash_profile file. Notice that you might not have a .bash_profile yet. If you do, it is important to make a backup copy of it by running the following commands at the command line:

cd
cp .bash_profile .bash_profile.bak

If you get a message stating that .bash_profile does not exits, do not worry. We will create one now.

Start your text editor and open the .bash_profile file. For example, to use nano, a simple text editor included with macOS, run the following in a Terminal window:

cd
nano .bash_profile

This will create .bash_profile if it still does not exist. Add the following line at the end of file, where you need to replace 3.x by the version of Python you have installed:

export PATH="$HOME/Library/Python/3.x/bin:$PATH"

Save the file, close the editor, and run the following command from the Terminal window:

source .bash_profile

This completes the installation of a basic SciPy stack. To test the setup, start Python 3 in the Terminal window by running the following command:

python3

To check that all packages we need were installed, execute the following lines at the >>> Python prompt. Notice that there will be no output, and, as long as there are no errors, the installation is correct:

import numpy
import scipy
import matplotlib
import IPython
import pandas
import sympy
import nose

You can now exit the Python shell by running the following statement at the prompt:

quit()

Let's now check that IPython is accessible from the command line. Run the following line from the Terminal window:

ipython

This will start IPython, an alternative Python shell with many added features that is required to run Jupyter. A message similar to the following will be printed:

Python 3.x.x (default, Apr  4 2017, 09:40:21) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]:

Exit Python by entering, the following command in the IPython prompt:

quit()

Let's now test if we can run the Jupyter Notebook. Run the following command from the Terminal window:

jupyter notebook

If all is correct, you will see a series of startup messages in the Terminal, and the Jupyter Notebook will eventually start on your browser. If you get an error message, you probably have an incorrectly configured PATH variable. Check the preceding tip box for instructions on how to fix it.

This finishes the installation of Python and the SciPy stack in macOS. Please proceed to the Setting up a virtual environment section.

Since Linux is an umbrella name for a number of distinct operating system configurations, there is no binary distribution that fits all possible Linux flavors.

All modern distributions of Linux come with Python pre-installed. In this recipe, we describe a setup procedure for a local Python 3 installation from source that works on two of the most popular Linux distributions, Ubuntu and Debian. The installation will be located in the user's home directory, and will not conflict with any pre-installed version of Python that exists in the system.

The installation procedure is broken down in the following stages:

• Installing Python 3
• Installing the SciPy Stack

Start by opening a Terminal window and running the following commands, one at a time. You will be required to enter the administrative password for your system which, on a personal computer, is usually your own login password:

sudo apt-get install build-essential
sudo apt-get install sqlite3 libsqlite3-dev
sudo apt-get install bzip2 libbz2-dev
sudo apt-get install libreadline-dev libncurses5-dev
sudo apt-get install libssl-dev tcl-dev tk-dev python3-tk
sudo apt-get install zlib1g-dev liblzma-dev

Next, download a source distribution of Python from the site https://python.org.

Make a note of the file name, which will look like the following: Python-3.x.x.tar.xz, where x.x is a pair of numbers that specify the build for this distribution. You should download the highest version available, which should be above 3.6.0.

Now, go to the directory where the downloaded file was saved and run the following command, replacing x.x with the corresponding build number of your file:

tar xvf Python-3.x.x.tar.xz

To build the distribution, execute the following commands, again replacing x.x with the correct build number:

cd Python-3.x.x
./configure --prefix=$HOME/python3
make

The build process will take a while to finish, depending on the configuration and speed of your system.

The following step is optional. If you want to run the battery of tests included with the source distribution, run the following command from the Terminal window:

make test

Depending on how fast your computer is, this may take a long time. At the end of the tests, a report of the build process will be generated. Do not worry if you get a few messages about skipped tests.

Next, install the code in its target directory, by running the following command:

make install

We now need to adjust the PATH environment variable. Start by making a backup copy of your shell configuration file by running the following commands:

cd
cp .bashrc .bashrc.python.bak

Start your text editor, open the .bashrc file, and add the following line at the end of the file:

export PATH=$HOME/python3/bin:$PATH

Save the file, close the editor and, back at the Terminal window, run the following command:

source ~/.bashrc

To test the installation, start Python from the command line by running the following command:

python3

If the installation is correct, Python will start and print information about the interpreter, as follows:

Python 3.x.x (default, Apr  4 2017, 09:40:21) 
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Check that the correct version of Python is being started by verifying that the version number 3.x.x coincides with that of the distribution you downloaded.

Exit Python by running the following command at the >>> Python prompt:

quit()

As a final test, run the following statement from the command line to check that Python's package manager was installed:

pip3 --version

This will print information about the version of pip3 that you are running. If no error message is issued, the setup was completed correctly and you can proceed to installing SciPy.

Open a Terminal window and enter each of the following commands in succession:

pip3 install --user numpy scipy matplotlib
pip3 install --user ipython jupyter PyQt5
pip3 install --user pandas sympy nose

Make a backup copy of the .bashrc file and open it with a text editor. For example, to use nano, run the following commands in a Terminal window:

cd
cp .bashrc .bashrc.scipy.bak
nano .bashrc

Add the following line at the end of .bashrc:

export PATH="$HOME/.local/bin:$PATH"

Save the file, close the editor, and run the following command from the Terminal window:

source .bashrc

Let's now test the installation. Start Python by running the following command in a Terminal window:

python3

Execute the following lines at the >>> Python prompt. There will be no output and, as long as there are no errors, the installation is correct:

import numpy
import scipy
import matplotlib
import IPython
import pandas
import sympy
import nose

Exit Python by running the following at the prompt:

quit()

Back at the Terminal prompt, run the following command:

ipython

This will start IPython, an alternative Python shell with many added features, and print a startup message. You can check that the Python shell is running the expected version of Python from the top line of the startup message. Exit IPython by entering, at the prompt, the following command:

quit()

Back at the Command Prompt, run the following from the command line:

jupyter notebook

If all is correct, the Jupyter Notebook will start on your browser after a while. You have now concluded the installation of Python 3 and the SciPy stack in Linux.

Sooner or later, you will need a package that is not installed by default in your Anaconda distribution.

If you are using the Anaconda distribution, this recipe describes the preferred method of installing and uninstalling packages. Anaconda is built on top of conda, a flexible and easy-to-use package manager.

Anaconda allows direct access to vast collections of add-on packages. For example, it makes it easy to install SciKits, which are SciPy packages developed by independent developers that are not part of the main distribution. A complete list of SciKits can be found at the following site: http://scikits.appspot.com/scikits.

This recipe assumes that you have a working installation of Anaconda. If you don't, follow the recipe for installing Anaconda for your operating system presented previously in this chapter.

As an example, lets install scikit-image, a package for image processing available at the site http://scikit-image.org:

To install scikit-image, first update conda and Anaconda by running the following two commands in your Terminal:

conda update conda
conda update anaconda

Now, scikit-image can be installed by running the following command at the system prompt:

conda install scikit-image

After the installation is finished, use a text editor to create a file called skimage_test.py containing the following code:

from skimage import data, io, filters

image = data.coins()
edges = filters.sobel(image)
io.imshow(edges)
io.show()

This code first constructs an image object from one of the sample images contained in scikit-image, which in this example is a photograph of a collection of old coins. It then applies a Sobel filter, which emphasizes the edges of objects present in the image. Finally, it displays the image of the detected coin edges.

Save the file and run the script by entering the following statement at the command line:

python3 skimage_test.py

Once the script runs, an image of the coin edges is displayed. Close the image to exit the script.

Let's now suppose that you don't need scikit-image any longer. It is easy to uninstall the package with conda using the following command:

conda uninstall scikit-image

If, after uninstalling, you attempt to run the skimage_test.py script, you will get an error message stating that Python can't import the skimage library.

Finally, what if a package is not available in Anaconda? In this case, it is still possible to install the package with pip, as outlined in the next section.

The main tool to set up packages in a standalone installation of Python 3 is the pip3 command. This command will fetch a package from the Python Package Index (PyPI), a standard public repository located at https://pypi.python.org/pypi.

PyPI is somewhat hard to navigate due to the large number of packages from different application areas that are available. A list restricted to packages related to SciPy is available at: https://www.scipy.org/topical-software.html.

The existence of a uniform package manager makes it is straightforward to add and remove packages to a Python installation.

As an example, let's say we want to install the Bokeh package, located at http://bokeh.pydata.org/.

Bokeh is a visualization library that targets modern browsers for presentation. It produces high-quality plots in a variety of formats for inclusion in web pages. To install Bokeh in macOS or Linux, all you need to do is to run the following command in a Terminal window:

pip3 install bokeh

To install on Windows, run the following command at the Command Prompt:

pip install bokeh

This will remotely access PyPI, fetch Bokeh and all its dependencies, build the package, and make it accessible in the current Python 3 installation. To test the package, use a text editor to enter the following code in the bokeh_test.py file:

import numpy as np
from bokeh.plotting import figure, show

xvalues = np.linspace(-np.pi, np.pi, 50)
y1values = np.sin(xvalues)
y2values = np.cos(xvalues)

p = figure(
 tools='pan,box_zoom,reset',
 title='Trigonometric functions',
 x_axis_label='x',
 y_axis_label='y' 
 )

p.line(xvalues, y1values,
 legend='y=sin(x)', line_color='blue')
p.circle(xvalues, y2values, 
 legend='y=cos(x)', fill_color='green', 
 line_color='green', size=3)

show(p)

print('Plot generated. Open file bokeh_test.html in your browser.')

This code uses the np.sin() and np.cos() NumPy functions to generate data points for the trigonometric functions sin(x) and cos(x). After that, it constructs a Bokeh figure object and adds two plots to the figure with the p.line() and p.circle() functions. Finally, the figure is displayed with a call to show(p).

Save the file in a convenient folder and open a Terminal window from the same folder. Run the following statement at the command line:

python3 bokeh_test.py

Running the script will create an output file named bokeh_test.html, with an HTML code that generates the figure described in the Python script. If you have a default browser defined for your system, Bokeh will automatically open the HTML file in the browser. Otherwise, you will need to open the file in the browser manually.

Notice that the graph displays control buttons at the top right. By clicking on the Pan button, the user can drag the graphed region with the mouse. With the Box Zoom tool, it is possible to draw a box on the graph, and the plot will be zoomed in on that box. Finally, Reset brings the figure back to its original state.

Let's say that Bokeh does not satisfy your needs and you do not want to keep it in your Python installation. The package can be easily uninstalled by issuing the following command in the Terminal, for Linux and macOS:

pip3 uninstall bokeh

On Windows, use the following command:

pip uninstall bokeh

The uninstaller will list the changes to be made and ask for confirmation. If you answer yes, the package and its dependencies will be removed from the system.

Setting up a virtual environments with conda is very easy, and is recommended even for small projects. Virtual environments are very handy.

This recipe assumes that you have a working installation of Anaconda. If you don't, follow the recipe for installing Anaconda on your operating system presented previously in this chapter.

One of the features of conda, the standard package manager used with Anaconda, offers the easy creation and management of virtual environments. We will show you three recipes presenting the typical uses of conda virtual environments.

Before diving into the examples, enter the following statement in the command line:

conda info

This will print information about the current Anaconda installation. To obtain a list of the existing conda environments, enter the following:

conda info --envs

When I run this, I get the following output:

# conda environments:
#
root                  *  /Users/luizmartins/anaconda

Right now, there is only the root environment, since this is a fresh Anaconda installation.

This recipe demonstrates how to set up an environment that is a clone of the current Anaconda installation. One possible use of this procedure is to set up an environment when we start the development of a new project.

This recipe assumes that you have a working installation of Anaconda. If you don't, follow the recipe for installing Anaconda on your operating system presented previously in this chapter.

1. Start by opening a Terminal window and running the following commands:

conda update conda
conda update anaconda

These commands update both conda and Anaconda to the most recent versions. This is always recommended before creating a new environment.

2. Next, run the following line to create a new environment named myenv:

conda create --name myenv

3. You will be asked for confirmation and conda will then create the new environment. We now need to activate the new environment. For Linux and macOS, run the following command in the terminal:

source activate myenv

4. In Windows, run the following:

activate myenv

5. Notice that the command line changes to reflect the active environment, as shown in the following example:

(myenv) computer:~ username

6. To confirm that the new environment was created, we can execute the following command:

conda info --envs

7. This command now produces the following output:

# conda environments:
#
myenv                 *  /Users/username/anaconda/envs/myenv
root                     /Users/username/anaconda

Notice that the currently active environment is marked with an asterisk.

8. We can now install packages in the active environment without interfering with the original Python distribution. For example, to install the csvkit package, we use code in the following example:

conda install csvkit

9. When you are done working in the environment, it is necessary to deactivate it. In Linux and macOS, this is done with the following command:

source deactivate

10. In Windows, the command to do so is the following:

deactivate

11. If you decide you don't need the myenv environment any longer, it can be deleted with the following command:

conda remove myenv --all

Note that you cannot remove the currently active environment.

We often need to run code that is not compatible with a particular version of a package. In this situation, it is useful to use conda to find out what version of a package is currently installed. For instance, to get information about which version of the numpy package is being used at the moment, we can execute the command shown as follows at the system prompt:

conda info numpy

On my computer, this outputs the following information:

numpy 1.12.1 py36_nomkl_0
file name   : numpy-1.12.1-py36_nomkl_0.tar.bz2
 name        : numpy
version     : 1.12.1

This recipe assumes that you have a working installation of Anaconda. If you don't, follow the recipe for installing Anaconda on your operating system presented previously in this chapter.

Let's now suppose that we have a legacy package that depends on an earlier version of NumPy. For example, let's assume we need version 1.7 of NumPy. We can check what versions of NumPy are available in the Anaconda repository with the following command:

conda search numpy

Running this command will produce a long list, in which we find the following information:

...  
1.7.1                    py33_0  defaults
...
1.7.1                    py33_2  defaults
...

conda create --name numpy17 python=3 numpy=1.7

The following NEW packages will be INSTALLED:
numpy:      1.7.1-py33_2 
openssl:    1.0.1k-1
pip:        8.0.3-py33_0 
python:     3.3.5-3
readline:   6.2-2
 ...

source activate numpy17

activate numpy17

source deactivate numpy17

deactivate numpy17

Another situation that comes up often is the need to test a package in a different version of Python. conda makes it easy to create a suitable environment. In this recipe, we will show you how to create and use environments with version 2.7 of Python.

This recipe assumes that you have a working installation of Anaconda. If you don't, follow the recipe for installing Anaconda on your operating system presented previously in this chapter.

1. Create the new environment with the following command:

conda create --name python27 python=2.7

conda will display information about the changes and ask for confirmation. Go ahead and enter yes to accept the environment creation.

2. To activate the environment on Linux and macOS, use the following command:

source activate python27

3. On Windows, the command to activate an environment is as follows:

activate python27

4. Let's now test the new environment. Start Python in the usual way after activating the environment. Notice that in the startup message, it should state that version 2.7 of Python is being run. Now run the following statement in the Python prompt:

zip([['a','b'],[1,2]])

5. This will produce the following output:

[('a', 1), ('b', 2)]

6. This confirms that we are running Python 2, since in Python 3 the zip() function returns an iterator object instead of a list. To leave the Python shell, run the following in the Python prompt:

quit()

7. When done with the environment, deactivate it by running the following command on Linux and macOS:

source deactivate python27

8. On Windows, run the following:

deactivate python27

A virtual environment is a clone of a Python installation stored in a local folder. The Python executables and packages are not really copied, but symbolic links to the original files are created and environment variables are adjusted to set up a consistent Python filesystem. Once activated, a virtual environment will install packages in local directories without modifying the global Python installation.

A virtual environment is also the recommended setup for development in Python. With a virtual environment, it becomes easy to isolate the packages required to install the software under development.

1. To create the new environment in the myenv directory, use the following command in Linux and macOS:

python3 -m venv myenv

2. In Windows, the command to be used is as follows:

python -m venv myenv

This creates an environment in the myenv subdirectory of the current directory. The -m switch is used because venv is a Python module that is being executed.

3. To switch to the new environment on Linux and macOS, run the following in the command line:

source myenv/bin/activate

4. On Windows, we use the following command:

myenv\bin\activate

5. If we need to install a package in the new environment, we use pip or pip3 as usual. For example, to install a molecule package using pip3, we execute the following command:

pip3 install molecule

6. Optionally, run the following command to create a requirements file:

pip3 freeze > requirements.txt

This will generate a report listing the packages required by the current environment and store the information in the requirements.txt file. This file can be distributed with the package being developed to facilitate installation by users that use pip.

7. When done working in the environment, deactivate it by running the following command:

deactivate

8. If we no longer need an environment, we can delete it as follows. First, activate the environment using the following commands:
   On Linux and macOS, use the following:

source myenv/bin/activate

On Windows, use the following:

myenv\bin\activate

9. Create a list of all packages in the environment with the following command:

pip3 freeze > requirements.txt

10. Remove all packages installed in the environment with the following command:

pip3 uninstall -y -r requirements.txt

11. The -y switch prevents pip from asking for confirmation to uninstall each individual package. Next, deactivate the environment by running the following command:

deactivate

12. Finally, remove the environment directory with the following commands:

On Linux and macOS use the following:

rm -rf myenv

On Windows use the following:

rmdir /s /q myenv

Executing a program from a text file is the most time-honored approach to running computer code. Since Python is an interpreted language, text files meant to be run with Python are called scripts. Scripts are an easy way to share and distribute your programs, since all code is encapsulated in a number of files that can be easily copied to another user's computer.

To follow this recipe, you need a text editor and and a Terminal window. The Terminal window must be open on the directory where your script is saved.

Running a script in Python is only a matter of typing the code with a text editor and running it using the Python interpreter. Enter the following sample code into the text editor:

import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm

plt.switch_backend('Qt5Agg')

fig = plt.figure()
ax = fig.gca(projection='3d')
ax.view_init(elev=25, azim=65)
xvalues = np.linspace(-4, 4, 40)
yvalues = np.linspace(-4, 4, 40)
xgrid, ygrid = np.meshgrid(xvalues, yvalues)
zvalues = xgrid**3 * ygrid + xgrid * ygrid**3
ax.plot_surface(xgrid, ygrid, zvalues, cmap=cm.coolwarm, 
                linewidth=0, antialiased=True)

plt.show()

Save the script in a file named script_test.py. Open a Terminal window on the directory where the script was saved and execute the following command from the system prompt:

python3 script_test.py

Running the script will produce a three-dimensional plot similar to the one displayed in the following screenshot:

The image displayed is interactive, and can be rotated and panned with the mouse. It is also possible to save the image to the disk. Notice that the script is suspended while the image is being displayed. To continue execution of the script, simply close the image. In our example, this will cause the script to end.

If your main goal in using SciPy is to do data exploration and analysis or scientific computations, Jupyter provides an ideal interactive environment. Using Jupyter, we can integrate computations, graphs, formatted text, and even more sophisticated media. Essentially, anything that can be inserted in a web page can be handled by Jupyter.

This recipe assumes that you have a working installation of IPython and Jupyter. If you followed one of the recipes in this chapter to set up Anaconda or a standalone installation of the SciPy stack, you have all you need.

The following steps demonstrate how to start Jupyter and create a new notebook:

1. Open a command window on the directory where you want your notebook files stored.
2. Start Jupyter by running the following command in the Terminal window:

jupyter notebook

3. After a few moments, the notebook will open in your web browser. The notebook starting page is known as the dashboard, and is shown in the following screenshot:

4. To create a new notebook, click the New button at the top right and select Python 3 from the menu. The following screenshot shows a newly created notebook:

Code in a notebook is entered in an execution cell, which is surrounded by a green border when active. To get a taste of what working with the Jupyter notebook feels like, click on an execution cell in the notebook and enter the following code:

%matplotlib inline 
import numpy as np
import matplotlib.pyplot as plt

With the mouse cursor still in the same execution cell, press Shift + Enter to run the cell. The preceding code displayed the magic %matplotlib inline first to tell Jupyter that we want to display plots in the notebook itself, and the next two lines imported NumPy and pyplot (an interactive plotting library provided by matplotlib).

After running the cell, the cursor automatically moves to the next cell. Enter the following code in this cell:

from scipy.stats import norm, binom
n, p = 100, 0.5
mean = n * p
sdev = np.sqrt(n * p * (1-p))
sample = np.array([binom.rvs(n, p) for _ in range(1000)])
xvalues = np.linspace(mean-3*sdev, mean+3*sdev, 200)
yvalues = norm.pdf(xvalues, loc=mean, scale=sdev)
hist = plt.hist(sample, normed=True,
                color='red', lw=3, ls='dotted', alpha=0.5)
plt.plot(xvalues, yvalues, color='blue', lw=2)
plt.title('Coin toss simulation, $n={}$, $p={:5.2f}$'.format(n, p))
plt.xlabel('Number of heads')
plt.ylabel('Frequency')
None

This code simulates 100 tosses of a fair coin. The simulation is repeated 1,000 times and the results are stored in the array sample. Then a histogram of the results is plotted, together with a normal approximation, according to the central limit theorem. Pressing Shift + Enter to run the cell will produce a plot of the histogram, representing the simulation and the theoretical normal approximation of the distribution of the number of heads in the coin tosses.

Spyder—which is an acronym for Scientific PYthon Development EnviRonment—is an IDE specifically designed for Python and SciPy. It provides an out-of-the box solution for developing projects that use NumPy, SciPy, pandas, and other scientific or data-oriented Python libraries.

Spyder is included in the default Anaconda distribution, so, if you are using Anaconda, you already have Spyder.

If not, you can install it with pip using the following commands:

pip3 install PyQt5
pip3 install spyder

Note that depending on your configuration, you may already have PyQt5 installed, but attempting to install it again will do no harm. pip3 will only tell you that the package is already present.

The first step is to start Spyder by running the following command from the command line:

spyder

After a few moments, the Spyder home window will appear on your screen, displaying an environment similar to the one shown in the following screenshot:

The window shown contains the following panes:

• On the left side of the window is the the Editor pane, where you enter code.
• The top right pane is the inspector pane. This pane is used to display information about code objects.
• The bottom right pane is the console pane. Notice that you can choose between an IPython console, a plain Python console, and the History Log.

To test the running code in Spyder, remove all text from the Editor window and type in the following code:

import numpy as np
import matplotlib.pyplot as plt
from scipy.special import eval_chebyt

fig = plt.figure()
xvalues = np.linspace(-1, 1, 300)
nmax = 5
for n in range(nmax+1):
   yvalues = eval_chebyt(n, xvalues)
   plt.plot(xvalues, yvalues)
plt.title('Chebyshev polynomials $T_n(x)$ for $n=0,\\ldots,{}$'.format(nmax))
plt.axhline(0, color='gray')
plt.axvline(0, color='gray')
plt.xlabel('$x$')
plt.ylabel('$T_n(x)$')

print('Displaying graph')
plt.show()

This code first imports NumPy, pyplot, and the special eval_chebyt function, which computes the values of Chebyshev polynomials of the first kind. Chebyshev polynomials are a family of orthogonal polynomials that are important in the design of some numerical methods. Then, the code defines a figure object, stored in the variable fig. Then it computes arrays containing values of the first six Chebyshev polynomials and plots them in the figure, which is then displayed after the formatting of the title and labels on the graph.

To run the code, go to the Spyder menu and select Run-Configure. The following window with configuration options will pop up on the screen:

In the Console section, choose Execute in a new dedicated Python console, and leave all other options with their default values. Click OK to dismiss the box. Now, back in the main Spyder window, choose Run-Run on the menu, or press F5. The script will be launched and, after a few seconds, a window with a plot of the Chebyshev polynomials will be displayed. Closing the graph window will continue the execution of the script.

Spyder is a powerful yet simple to use IDE. To explore the features of Spyder, the reader is encouraged to explore the software's official documentation: https://pythonhosted.org/spyder/.

For projects of moderate to large size, using a full-fledged Integrated Development Environment (IDE) is essential for efficient code production. An IDE might look, at first, like an imposing tool due to its complexity. A typical modern IDE integrates tools for editing, project management, building and running code, debugging, profiling, packaging for distribution, and integration with version control systems. As with any sophisticated tool, there is a learning curve associated with mastering a new IDE. The gains in productivity in the software development process, however, easily outweigh the time spent learning how to use an IDE.

PyCharm is an IDE developed by JetBrains, providing a well-rounded collection of tools designed with Python development in mind. To experiment with PyCharm, go to the following address: https://www.jetbrains.com/pycharm/.

Click on Download Now, and you will be given a choice between the Professional Version and the Community Version. Choose the one that best suits you. The Community Version has limited capabilities but can be used for free. The Professional Version requires the purchase of a license.

After the download has finished, run the downloaded installer and follow the instructions.

Start PyCharm according to the directions for your operating system. Depending on the version of PyCharm you are running, you may be asked to enter licensing information. After that, the startup window will show up on your screen, as shown in the following screenshot:

PyCharm requires the definition of a Project containing all files related to a particular development project. This may seem like overkill if we just want to run a single script, but PyCharm's project structure is very flexible. It does not require the setting up of a main entry point, and allows for several running configurations. This way, it is easy to run and debug individual scripts independently.

Click on Create New Project on the startup window, and the New Project window will pop up, as shown in the following screenshot:

Select the following options in the New Project window:

In the Project type pane at the left, select Pure Python.

In the Location field, enter the directory where you want the project to be saved. You can use the button at the right, labeled with ..., to choose the directory visually. If the directory does not exist, it will be created for you. In this example, we are using pycharm-test as the directory for our project. This will also be the name of the project in PyCharm.

Choose the Python interpreter for the project. For the purposes of this book, this should be the interpreter in the Python environment that was installed according to the instructions given previously in this chapter.

Click the Create button, and the project creation process will begin.

If this is the first time you are running PyCharm, you may notice that, at the bottom right of the PyCharm window, there is an indicator stating that there are processes running in the background:

This indicator appears because PyCharm is indexing the Python packages currently in your environment. This index is used by several PyCharm components, including Intellisense, syntax highlighting, and syntax checker. The indexing process may take up to a few minutes, depending on how fast your computer is and how many packages are installed in the Python tree. You can continue to use PyCharm during the indexing process, but you may observe degraded responsiveness. Everything should go back to normal once the indexing process is finished.

Let's now take a brief tour of the main PyCharm window, shown in the following screenshot:

Right now, no files are being edited, and the right pane does not show any code. The left pane is currently showing the Project Navigator, from which all project components can be easily accessed. Right now, the project has no files.

Click on the arrow to the left of External Libraries. This gives you access to the Python tree corresponding to the Python interpreter that is being used in this project. Notice that the interpreter itself is listed right at the top of the list. You can use this pane to navigate through the Python installation, and open the source of any installed module in the editor .

Another useful component of the PyCharm window is the tool menu, represented by a little box on the extreme bottom left of the screen. Hover your mouse pointer over the box, and the menu shown in the following screenshot will be displayed:

This menu contains shortcuts to frequently used tools. If you work a lot with PyCharm, this menu is one of the best time savers offered by the IDE.

Let's now create some code and run it. Right-click on the project name, pycharm-test, on the Project Navigator and select New from the context menu. Then select Python File in the submenu that pops up. Enter the name of the file in the appropriate field, without the .py extension. Click OK and the file will be created and opened in the Editor pane.

Enter the following code in the pycharm-test file. To get a feel for PyCharm's coding features, it is recommended that you type the code yourself, instead of copying and pasting it to the Editor window:

def collatz_sequence(n):
    collatz_seq = [n]
    while n != 1:
        if n % 2 == 0:
            n //= 2
        else:
            n = 3 * n + 1
        collatz_seq.append(n)
    return collatz_seq


while True:
    n_input = input("Enter starting value (enter 0 to exit): ")
    try:
        n = int(n_input)
    except ValueError:
        print('Invalid integer: {}'.format(n_input))
        continue
    if n < 0:
        print('n must be positive')
    if n == 0:
        break
    cseq = collatz_sequence(n)
    print('Collatz sequence for n={}:\n{}'.format(n, cseq))

print('Thanks for playing, I hope you had fun.')

Notice the following as the code is typed:

• PyCharm will offer suggestions as you type the code. PyCharm analyzes your code on the fly, and searches the libraries and code you have already written for sensible suggestions as to what you are trying to type. To accept a suggestion, simply press Enter.
• If there are several alternatives, you can move up and down the list of suggestions with the arrow keys.
• Syntax errors are highlighted as they appear, and you can correct them before running the code.
• Suspicious code is also indicated. For example, if you define a variable or import and do not use it, PyCharm will warn you.

This code lets the user experiment with the famous Collatz problem, which is is an apparently simple problem that has so far resisted any approach anyone has tried. The problem consists of showing that a sequence constructed from a simple mathematical rule will always be finite, independently of the starting value of the sequence. Here, we simply use it as a example of code to be tested, but interested readers can read about the history of the problem at the following site: https://en.wikipedia.org/wiki/Collatz_conjecture.

To run the code, save the file and select pycharm-test on the Project Navigator. Right-click on pycharm-test and select Run pycharm-test. The script will start and, if there are no syntax errors, you will be prompted for a number on the PyCharm console, shown as follows:

Enter starting value (enter 0 to exit): 

Enter a positive integer and press Enter. The script will compute the corresponding Collatz sequence and print it on the screen:

Enter starting value (enter 0 to exit): 7
 Collatz sequence for n=7:
 [7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1]
 Enter starting value (enter 0 to exit): 22
 Collatz sequence for n=22:
 [22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1]

After you are done experimenting with the script, enter 0 in the prompt line and the program will stop:

Enter starting value (enter 0 to exit): 0
Thanks for playing, I hope you had fun.
Process finished with exit code 0

Let's now use this code to experiment with the debugger. In the text editor, change the code that reads n = 3 * n + 1 to n = 2 * n + 1 and save the script. Run the code and enter a positive integer in the prompt. You will notice that no output is produced, and the script seems to be running in a loop. On the PyCharm menu, select Run-Stop pycharm-test. The script will be stopped and a KeyboardInterrupt exception will be reported in the console window. Stopping the script this way is equivalent to pressing Ctrl + C at the keyboard.

Let's set up a breakpoint to debug the code. We want to stop the script right after it enters the loop, at the line:

n_input = input("Enter starting value (enter 0 to exit): ")

To do this, click on the gutter at the left of the line, where we want to insert a breakpoint. The newly inserted breakpoint will be highlighted, as shown in the following screenshot:

Let's now start the script in debugging mode, by right-clicking pycharm-test on the Project Navigator and selecting Debug pycharm-test. The script will start and break at the line that asks for user input. Select Run-Step over on the menu, or press the F8 key, to run the next line in the script. When prompted, enter a positive integer and press Enter on the keyboard. At this point, the next line in the script will be highlighted:

try:

Continue to step over the code, until you get to the line:

cseq = collatz_sequence(n)

At this point, we want to go into the collatz_sequence() function to see what it is doing. To do this, select Run-Step into, or press F7. This will bring you into the code for collatz_sequence(). Now, continue to step over code (F8) until you can guess what is happening with the code.

You will notice that in the if statement inside the while loop, the n = 2 * n + 1 statement is being repeatedly executed. Notice what happens in the Editor window. After a few runs through the loop, the editor will show something like the following screenshot:

Notice that as you step over code, PyCharm displays an update about the contents of the collate_seq list. You will notice that this list keeps growing and, if you let the code run long enough, you will eventually get an exception due to a memory fault.

So, what is happening here? When the code gets to the if statement, it does the n % 2 == 0 test. This returns True only when n is even. If n is odd, the code falls through the else clause, where n is updated by the n = 2 * n + 1 statement. As a result, the updated value of n will again be an odd number and, the next time the loop body is executed, the code will again fall into the else clause. It is clear that this leads to an infinite loop.

Change the n = 2 * n + 1 line back to n = 3 * n + 1 to fix the code, and run it again to make sure everything is back as it should be.

The attentive reader will notice that, with the correct line of code, n = 3 * n + 1, if a particular value of n is odd, the next value of n will be even. Then, there will be a sequence of steps in which the number n is halved in the n //= 2 statement, until we get another odd number. One of the difficulties with the Collatz problem is that it seems that the number of times that n is halved is quite unpredictable, even though the code is completely deterministic. The reader might want to investigate the problem by him or herself, but be warned, this can be quite a rabbit hole!

In the preceding examples, we have barely scratched the surface of all features offered by PyCharm. The reader is invited to try using PyCharm on one of his/her projects, and to read the documentation to learn all that is available.