Chapter 7. Building and Distributing NumPy Code

In a real-world scenario, you will be writing an application with the intentions of distributing it to the World or reusing it on various other computers. For this purpose, you would like your application to be packed in a standard way so that everyone in the community understands and follows. As you will have noticed by now, Python users mainly use a package manager called pip to automate the installation of modules created by other programmers. Python has a packaging platform called PyPI (Python Package Index), which is an official central repository for more than 50,000 Python packages. Once the package is registered in PyPi aka Cheese Shop, other users across the world can install it after configuring it with package management systems such as pip. Python comes with a number of solutions to help you to build your code ready for distribution to the Cheese Shop and, in this chapter, we will focus on two such tools, setuptools and Distutils...

Introducing Distutils and setuptools

Before we begin, first let us understand what these tools are and why we prefer one over another. Distutils is a framework that comes by default with Python, and setuptools builds over the standard Distutils to provide enhanced functionalities and features. In a real-world scenario, you will never use Distutils. The only case where you might want to use Distutils alone is where setuptools is unavailable. (A good setup script should check for the availability of setuptools before proceeding.) In most cases, users will be better off installing setuptools as most packages nowadays are built over them. We will be using setuptools for building Cython code in the following chapters; hence, for our purpose, we will be installing setuptools now and using it extensively from now on.

Next let us start by installing the required tools to build our first dummy (but working) installer. After we have got our installer working, we will dive into more functionalities...

Preparing the tools

To install setuptools on your system, you need to first download ez_setup.py on your system from https://pypi.python.org/pypi/setuptools and then execute this from your command prompt as follows:

    $ python ez_setup.py

To test the installation of setuptools, open the Python shell and type the following:

> import setuptools 

If the preceding import does not give any error, then we have successfully installed setuptools.

Building the first working distribution

All of the tools we mentioned previously (setuptools, Distutils and numpy.distutils) are centered around the function setup. To get an understanding of most packaging requirements, we will look into a simple setup function and then study a full-fledged installer. To create a basic installer, we need to call the setup function with metadata about the package. Let's call our first package py_hello, which has just one function greeter, and just prints a message when called. The package can be downloaded from the Bitbucket repository at https://bitbucket.org/tdatta/books/src/af376df081ef/python/simple_setup/?at=master The project directory structure for the project looks like the following:

py_hello 
├── README 
├── MANIFEST.in 
├── setup.py 
├── bin 
│   └── greeter.bat 
└── greeter 
    ├── __init__.py 
    ├── greeter.py 

Let's look at some standard files here:

• README-This file is used to store information...

Testing your package

It is very important that the package you are building should work/install without any problem on a user's computer. Hence, you should spend time testing the package. The general idea behind testing installation is to create a virtualenv and try to install the package or use another system altogether. Any error encountered at this stage should be removed and the author should try to make sure that the exceptions are easier to follow. Exceptions should also try to provide the solution. Common mistakes at this stage would be:

• Assumptions about preinstalled modules and libraries.
• Developers may forget to include dependencies in the setup file. This mistake will be caught if you use a new virtualenv to test the installer.
• Permissions and elevated rights requirements.
• Some users may have read-only access to the computer. This could be easily overlooked as most developers do not have this case in their own machines. This problem should not occur if the provider of the package...

Distributing your application

Once all the development for your module/application is complete and you are ready with your complete working application and setup file, the next task will be to share your hardwork with the World to benefit others. The steps to release it to the world by using PyPI are quite straightforward. The first thing you need to do as an author of a package is to register yourself. You can do it directly from the command line as follows:

    $ python setup.py register
    running register
    running egg_info
    ....
    ....
    We need to know who you are, so please choose either:
     1. use your existing login,
     2. register as a new user,
     3. have the server generate a new password for you (and email it to you), or
     4. quit
    Your selection [default 1]:

Tip

This process will fail if you do not have proper metadata information of any file missing in your setup.py. Make sure you have setup.py working...

Summary

In this chapter, we introduced the tools used for packaging and distributing applications. We first looked at a simpler setup.py file. You looked into the attributes of the function setup and how these arguments link up to the final installer. Next we added-NumPy related code and added some exception handling code. Finally, we built the installer and learnt how to upload it on the Cheese Shop (the PyPI website). In the next chapter, you will be looking at ways to further speed up your Python code by converting parts of it into Cython.