Python Essentials

4.8 (9 reviews total)
By Steven F. Lott
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Getting Started

About this book

Python Essentials provides a vital tour of the most critical features of Python. Starting with setup and installation, you will soon dive into exploring built-in-library types, Python's rich collection of operators and built-in functions, variables, assignment and scoping rules.

From this foundation, you will explore functions, a crucial aspect of any programming language, including considerable sophistication in defining parameters to a function and providing argument values. Explore advanced functional programming using generator expressions, comprehensions, and generator functions. Handle file input and output using web services and context managers, exception handling and explore wider, popular frameworks.

Through this concise and practical guide, you will explore all you need to know to leverage this powerful, and industry-standard, programming language.

Publication date:
June 2015
Publisher
Packt
Pages
298
ISBN
9781784390341

 

Chapter 1. Getting Started

Python comes on some computers as part of the OS. On other computers, we'll need to add the Python program and related tools. The installation is pretty simple, but we'll review the details to be sure that everyone has a common foundation.

Once we have Python, we'll need to confirm that Python is present. In some cases, we'll have more than one version of Python available. We need to be sure that we're using Python 3.4 or newer. To confirm that Python's available, we'll do a few interactions at Python's >>> prompt.

To extend our foundation for the remaining chapters, we'll look at a few essential rules of Python syntax. This isn't complete, but it will help us write scripts and learn the language. After we've had more chances to work with simple and compound statements, the detailed syntax rules will make sense.

We'll also look at the Python "ecosystem", starting with the built-in standard library. We'll emphasize the standard library throughout this book for two reasons. First, it's immense—much of what we need is already on our computer. Second, and more important, studying this library is the best way to learn the finer points of Python programming.

Beyond the built-in library, we'll take a look at the Python Package Index (PyPI). If we can't find the right module in the standard library, the second place to look for extensions is PyPI—https://pypi.python.org.

 

Installation or upgrade


To work with Python on Windows, we must install Python. For Mac OS X and Linux, a version of Python is already present; we'll often want to add a newer version to the preinstalled Python.

There are two significantly different flavors of Python available:

  • Python 2.x

  • Python 3.x

This book is about Python 3.4. We won't cover Python 2.x at all. There are several visible differences. What's important is that Python 2.x is a bit of a mess under the hood. Python 3 reflects some fundamental improvements. The improvements came at the cost of a few areas where the two versions of the language had to be made incompatible.

The Python community is continuing to keep Python 2.x around. Doing this is a help to people who are stuck with old software. For the most part, developers are moving forward with Python 3 because it's a clear improvement.

Before we get started, it's important to know if Python is already installed. The general test to see if Python is already installed is to get an OS command prompt. For Windows, use Command Prompt; for Mac OS X or Linux, use the Terminal tool. We'll show Mac OS X prompts from the Mac OS X Terminal. It looks like this:

MacBookPro-SLott:~ slott$ python3
Python 3.3.4 (v3.3.4:7ff62415e426, Feb  9 2014, 00:29:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

We've shown the OS prompt MacBookPro-SLott:~ slott$. We entered the python3 command, which is typical for Linux and Mac OS X. In Windows, we'll often enter just python. The response was three lines of introduction followed by the >>> prompt. Enter exit and hit return to get some useful advice on how to leave Python. This example showed Python 3.3, which is a little out of date. An upgrade isn't required.

Some kind of "command not found" error from the OS means we don't have any Python, so we'll need to do an install.

If we get a Python message that starts with something like "Python 2.7.6", we'll need to do an upgrade.

The next section covers Windows installations. After that, we'll look at Mac OS X and then we will see Linux upgrades. In some cases, we may develop software on Windows desktop computers, but the ultimate destination is a large, centralized Linux server. The Python files can be the same between these two environments, so having Python on multiple platforms won't be very complex or confusing.

Installing Python on Windows

Python runs on many versions of Windows. There are some older, less-widely-used versions of Windows without an actively supported version of Python. For example, Windows 2000 is not supported.

The general procedure for installing Python is quite simple. We'll download an installer and do some preparation. Then we'll start the installer. Once that's finished, we'll be up and running.

To find the installer, start here:

https://www.python.org/downloads/

The web server should detect your OS and provide a big button with some variation of "Download Python 3.4.x" on it. Click on this button to start the download.

To look at the choices available, the https://www.python.org/downloads/windows/ path provides all of the actively-supported versions of Python. This will show a long list of older versions. There are two installers available:

  • The Windows x86 MSI installer

  • The Windows x86-64 MSI installer

If we have a very old computer, we might need the 32-bit version. Most modern computers will have a 64-bit CPU. When in doubt, 64-bit is the assumption to make.

Double-click the .msi file to start running the installer. This starts with a question about installing Python for yourself or for all users. If you have appropriate privileges, the all users option is appropriate. On a shared computer, without appropriate privileges, you'll have to install it for yourself only.

The second page will ask for an installation directory. Be careful about the path that you choose for the installation, and avoid spaces in filenames.

Tip

Do not install Python into directories with spaces in their names. Avoid names such as "Program Files" and "My Documents". The space may cause problems that are difficult to diagnose.

Install Python into a simple directory with a short, space-free name like C:\python34.

Spaces in filenames is not a general problem, but it is awkward when first starting. There are many ways to cope with spaces in filenames. When learning a new programming language, however, it's important to minimize the awkward problems so that we can focus on the important topics.

The next page will also show a menu of components that can be installed; it's easiest to request everything. There's no compelling reason to turn off any of the optional components. We'll be looking at the IDLE development tool, which requires the Tcl/Tk package, so it's important to be sure that this is part of the installation.

In many cases, the final option on this list updates the system environment variables to include Python on the PATH variable. This isn't enabled by default, but it can be helpful if you're going to write BAT files in Windows.

In addition to the basic Python interpreter, the Windows help installer is very helpful. This is a separate download and requires a quick installation. After we've installed this, we can use the F1 key to bring up all of the Python documentation.

Once Python is installed, the Using the Read-Evaluate-Print Loop (REPL) section will show how to start interacting with Python.

Considering some alternatives

We'll focus on a particular implementation of Python called CPython. The distinction we're making here is that Python—the abstract language—can be processed by a variety of concrete Python runtimes or implementations. The CPython implementation is written in portable C and can be recompiled for many operating systems. This implementation tends to be extremely fast.

For Windows developers, there's an alternative implementation called Iron Python. This is tightly integrated with the Windows .NET development environment. It has the advantage of working with Visual Studio. It has the disadvantage of being based on the Python 2.7 language.

Another choice Windows users have is to use Python Tools for Visual Studio (PTVS). This will allow you to use Python 3.4 from within Visual Studio. For developers who are used to Visual Studio, this might prove helpful.

Other Python implementations include Jython, Stackless Python, and PyPy. These alternatives are available for all operating systems, so we'll address these in the Looking at other Python interpreters section later.

Upgrading to Python 3.4 in Mac OS X

Python runs on all versions of Mac OS X. It turns out that Mac OS X relies on Python. However, it relies on Python 2.7, so we'll need to add Python 3.4.

The general procedure for installing Python on Mac OS X is quite simple. We'll download a disk image (.dmg) installer and do some preparation. Then we'll start the installer that's in the disk image. Once that's finished, we'll be up and running.

To find an installer, start here:

https://www.python.org/downloads/

The web server should detect your OS and provide a big button with some variation of "Download Python 3.4.x" on it. Click on this and download the .dmg file.

To look at the choices available, the https://www.python.org/downloads/mac-osx/ path provides all of the actively-supported versions of Python for Mac OS X. This will show alternatives for older versions of Python.

When the .dmg device becomes available after the download, double-click on the .mpkg installer file to start running the installer.

Clicking on Continue will step through the Read Me, License, Destination Select, and Installation Type windows. There's a Customize button that allows us to turn options on and off. We won't need to do this—the default installation is ideal.

We'll need to provide the username and password of a user who's authorized to administer this computer. This will not remove the existing Python that Mac OS X uses. It will add another version of Python. This means that we'll have at least two copies of Python. We'll focus on using Python 3, ignoring the built-in Python, which is Python 2.

To use Python 3, we have to enter python3 at the OS prompt in the Terminal window. If we have both Python 3.3 and Python 3.4, we can enter the even more specific python3.4 at the command prompt to specify which version of Python 3 we're using. Generally, the python3 command will be the latest-and-greatest version of Python 3. The python command—unadorned with a version number—will be the Python 2.x that Mac OS X requires.

Adding the Tkinter package

Python relies on a library named Tkinter to provide support for writing programs with a GUI. This package relies on Tcl/Tk. The details can be found here:

https://www.python.org/download/mac/tcltk/

The summary of this is that we need to install version 8.5.17 or newer. See https://www.python.org/download/mac/tcltk/#activetcl-8-5-17-0. This will provide a graphic environment that Python will use. We must install Tcl/Tk in order for the tkinter package to work.

After we download the .dmg file and open the .pkg file, we'll see this window:

We'll be looking at the IDLE development tool, which requires tkinter. Consequently, this additional installation is essential.

We can avoid this extra download if we avoid using tkinter. Some developers prefer to use the Active State Komodo editor as their development tool; this does not require Tcl/Tk. Also, there are numerous add-on GUI frameworks that don't require tkinter.

Upgrading to Python 3.4 in Linux

For Linux, the latest Python may already be installed. When we enter python3, we may see that we already have a useful version available. In this case, we're ready to roll. In some cases, the OS will only have an older Python (perhaps older than 2.7) installed. In this case, we'll need to upgrade.

For Linux distributions, there are two paths for upgrading Python:

  • Installing prebuilt packages: Many distributions have appropriate packages already available. We can use a package manager (such as yum or RPM) to locate and install the necessary Python package. In some cases, there will be additional dependencies, leading to a cascade of downloads and installs. Since Python 3.4 is relatively new, there may not be very many prebuilt packages for your particular Linux distribution. Details are available at https://docs.python.org/3/using/unix.html#on-linux.

  • Building from source: Most Linux distributions include the GNU C compiler. We can download the Python source, configure the build script, and use make and make install to build Python. This may require upgrading some Linux libraries to assure that your Linux installation has the required support for Python 3.4. The installation steps are summarized as ./configure, make, and sudo make altinstall. Details are available at https://docs.python.org/3/using/unix.html#building-python.

When we use altinstall, we'll end up with two Pythons installed. We'll have an older Python, which we can run using the python command. The python3 command will, generally, be linked to the latest version of Python 3. If we need to be explicit, we can use the python3.4 command to select a specific version.

As with the Mac OS X installation, adding the Python tkinter package is important. Sometimes, this is separate from the basic package. This may lead to upgrading Tcl/Tk, which may lead to some more downloads and installs. At other times, the Linux distribution has an up-to-date Tcl/Tk environment and nothing more needs to be done.

We can avoid the extra Tcl/Tk download if we avoid using tkinter. As mentioned earlier, many developers prefer to use the Active State Komodo editor as their development tool; this does not require tkinter. Also, there are numerous GUI frameworks that aren't based on tkinter.

 

Using the Read-Evaluate-Print Loop (REPL)


Once we have installed Python 3, we can do some minimal interaction with Python to assure ourselves that things are working. In the long run, we'll use a number of other tools to create Python programs. To start out, we'll interact directly on the command line.

Python's Read-Evaluate-Print Loop (REPL) is the foundation for Python programming. More sophisticated things—such as writing application scripts or web servers—are essentially the same as interaction with the REPL: the Python program reads statements from our application script file or web server script file and evaluates those statements.

This fundamental rule is one of the very appealing features of Python. We can write sophisticated scripts, or we can interact with the language in the REPL; the language is the same.

Confirming that things are working

To confirm that things are working, we'll start the Python interpreter from a command-line prompt. It might similar to like this:

MacBookPro-SLott:~ slott$ python3
Python 3.3.4 (v3.3.4:7ff62415e426, Feb  9 2014, 00:29:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

The details of getting to a command prompt vary from OS to OS. We've shown the Mac OS X Terminal tool in this example. We entered the python3 command to be sure we ran our new version of Python, not the built-in Python 2.

The introductory message lists four special-purpose objects that are incorporated into the interactive Python environment. There are two more, quit and exit, which are also available. These are only present in the REPL interactive environment; they cannot be used in programs.

We'll look at how we get help later in a separate section, Interacting with the help subsystem. The other objects, however, produce useful tidbits of information and are the ideal way to be sure things are working. Enter copyright, credits, or license at the >>> prompt to confirm that Python is working.

Doing simple arithmetic

The REPL loop prints the results of each statement, allowing us to work with Python interactively. To be clear on what this means, we should define what constitutes a statement in the language. We'll avoid the strict formality of the Python language definition and provide a quick, informal definition of the relevant statement type.

The Python language has 20 or so kinds of statements. An expression—by itself—is a statement. Unless the value of the expression is None, the REPL will show the value of the expression. We'll often use an expression statement to evaluate functions that perform input and output.

This simple expression statement allows us to do things such as the following at the Python >>> prompt:

>>> 355/113
3.1415929203539825

We can enter any arithmetic expression. Python evaluates the expression, and if the result isn't None, we'll see the result. We've shown the true division operator, /, in this example.

We'll look at the various data types and operators in Chapter 2, Simple Data Types. For the moment, we'll identify a few features of Python. We have numbers in a variety of flavors, including integers, floating point, and complex values. Most values will be properly coerced to add precision. Have a look at these examples:

>>> 2 * 3.14 * 8j
50.24j
>>> _ **2
(-2524.0576+0j)

The first expression computed a value that includes an integer, 2; a floating point value, 3.14; and a complex value, 8j. We used the * operator for multiplication. The result is complex, 50.24j.

The second expression uses the _ variable. This is a handy feature that's unique to the REPL. The result of each expression is implicitly assigned to this variable. We can use _ in an expression to refer to the result of the previous expression. This only works in the REPL; it's never a part of a script.

When we computed _ **2, we squared 50.24j. This is -2524.0576. Since the source value was a complex number, the result is also a complex value even though the imaginary component of that complex value is zero. This is typical of Python—the data types of the operand values generally dictate the data types of the result of the operator. When there are different kinds of numbers, values are coerced according to the rules we'll look at in Chapter 2, Simple Data Types.

There's one notable exception to the rule that the types of the operands match the type of the result. The true division operator, /, produces floating point results from integer operands. The floor division operator, //, on the other hand, reflects the types of the operands. For example:

>>> 355 / 113
3.1415929203539825
>>> 355 // 113
3

We have these two division operators so that we can unambiguously specify what kind of division we'd like to perform. It saves us from having to write extra code to explicitly coerce results.

Assigning results to variables

The simple assignment statement produces no visible output:

>>> v = 23

This will create the variable v and assign the value of 23 to it. We can check this by using a very small expression statement. The expression is just the variable name:

>>> v
23

When we evaluate a very simple expression, such as v, we see the value of the variable.

Python's REPL has far-reaching consequences. Perhaps the most important consequence is that almost all examples of Python programming are provided as if we're entering the code at the >>> prompt. The documentation for very complex and sophisticated packages will be written as though we're going to use that package interactively. In most cases, we'll be writing application programs; we won't really do very much at the >>> prompt. But the idea of cutting through the complexity to arrive at something that can be done interactively is pervasive throughout the Python community.

Using import to add features

One significant part of Python is the presence of a vast library of additional features. Using an external library means that the core language can be kept quite simple. We can import any additional features we need, avoiding the clutter and complication of unused features.

The import statement is used to incorporate additional functions, classes, and objects into a program or the interactive environment. There are a number of variations of this statement. For example, we might want to use some of the more sophisticated math functions. We can search the Python documentation and discover that these are defined in the math library. We can include and use them like this:

>>> import math
>>> math.pi
3.141592653589793
>>> math.sin( math.pi/6 )
0.49999999999999994

In this example, we imported the math library. We evaluated math.pi to see one of the constants defined in this library. We evaluated .The result was almost (but not exactly) 1/2.

This also shows us an important thing about floating point numbers—they're just an approximation. This has nothing to do with Python specifically—it's a general feature of digital computing. It's very important to emphasize this fact about floating point numbers.

Tip

Floating point numbers are only an approximation. They're not exact. They are not the abstract mathematical ideal of an irrational number with infinite precision.

We'll return to the topic of floating point numbers in Chapter 2, Simple Data Types. For now, we want to focus on external libraries.

One important library module that is part of Python is named this. To see the this module, enter import this at the >>> prompt, like so:

>>> import this

Another equally important module is antigravity.

>>> import antigravity

We'll leave the exploration of these modules as exercises for the reader. We don't want to spoil the fun! More handwaving explanation isn't as helpful as hands-on experience. See http://xkcd.com/413/ for more on this topic.

We'll summarize by noting that the name "Python" has much to do with Monty Python and nothing to do with serpents.

 

Interacting with the help subsystem


Python's interactive help utility provides a great deal of useful information about modules, classes, functions, and objects. The help system is an environment that is distinct from Python's REPL; it provides distinct prompts to make this clear.

There are three help modes, each with its unique prompt:

  • We'll see the help> prompt from the Python help environment. When we evaluate the help() function with no argument value, we'll enter Python's help environment. We can enter different subjects and read about various Python features. When we enter quit as a topic, we'll return to the REPL.

  • Using Windows, we'll see the -- More -- prompt: When we evaluate something like help(int) in a Windows environment, the output will be displayed using the MS-DOS more command. For more information, enter ? for help on how to page through the help() output. At the Windows command line, entering more /? will provide additional information on how the more command helps you page through a long file.

  • Using Mac OS X and Linux, we'll see the : prompt. When we evaluate the help() function with a specific argument value—for example, help(float)—in Mac OS X or Linux, we'll get output that's displayed using the less program. For more information on this, enter h for help while viewing the help() output. At the command prompt, enter less -? for more information on how the less program works.

There are additional ways to view the documentation available with Python modules. In IDLE, for example, there's a class browser and path browser that will show documentation about modules and files. This is based on the built-in help() function, but it's displayed in a separate window.

Using the pydoc program

Python includes the pydoc application that we use to view documentation. This application is something that we run from the OS command prompt. We do not use this from the Python >>> prompt; we use it from the OS prompt. While developing, we might want to leave a Terminal window open just to display module documentation.

The pydoc program has two operating modes:

  • It can show some documentation about a specific package or module. This will use an appropriate program (more on Windows, but otherwise less) to display documentation for the given object. Here's how we can display documentation on the math module:

    MacBookPro-SLott:~ slott$ python3 -m pydoc math
    
  • It can start a documentation web server. This will start a server (and also start a browser) to look at Python module documentation. When we use it, we'll have a session that looks like this:

    MacBookPro-SLott:~ slott$ python3 -m pydoc -b
    Server ready at http://localhost:50177/
    Server commands: [b]rowser, [q]uit
    server> q
    Server stopped
    

The second example will start a web server as well as a browser. The browser will show the pydoc-produced documentation. This is derived from the module and package structure as well as the documentation strings embedded in the Python code. When we were done reading the documentation, we entered q to quit the web server.

When we write Python packages, modules, classes, and functions, we can (and should) provide the content for pydoc/help() documentation. These documentation strings are part of our programming, and are as important as having programs that work correctly. We'll look at this embedded documentation in Chapter 14, Fit and Finish – Unit Testing, Packaging, and Documentation.

 

Creating simple script files


While we can use all of Python from the REPL, this is not a good way to produce a final application. Most of what we do with Python will be done via script files. We'll look at script files in detail in Chapter 12, Scripts, Modules, Packages, Libraries, and Applications. For now, we'll look at a few features.

A script file has to follow a few rules:

  • The content must be plain text. While ASCII encoding is preferred by some, Python 3 can easily handle UTF-8 and most OS-specific variations such as Mac OS Roman or Windows CP-1252. A portable encoding like UTF-8 is strongly suggested.

  • Python can cope with Mac OS X, Linux newline (\n), as well as Windows CR-LF (\r\n). Only a few Windows tools, such as Notepad, insist on CR-LF line endings; most other programming editors discern the line endings flexibly. Unless you really must use Notepad, it's often best to use Unix-style newline line endings.

  • The filename should be a legal Python identifier. This is not a requirement, but it gives us considerable flexibility if we follow this suggestion. The Language Reference Manual, section 2.3, provides the details of what constitutes an identifier. A summary of these rules is that identifiers must begin with a letter (or a Unicode character that normalizes to a letter) or _. It continues with letters, digits, and the _ character. What's important is that we should avoid characters that are Python operators or delimiters in filenames. In particular, we should avoid the hyphen (-), which can become a problem in some Python contexts. OS filenames have much more flexible rules than Python identifiers, and the OS has ways to escape the meaning of OS-related punctuation; we are happiest when we limit our filenames to valid Python identifiers – letters, digits, and _.

  • The filename extension should be .py. Again, this is not required, but it is very helpful to follow this rule.

For example, we'll try to focus on names such as test_1_2.py. We can't as easily use a file named test-1.2.py; the base name isn't a valid identifier—this name looks like a Python expression. While the second name is acceptable for a top-level script, it won't work as a module or package.

We'll look at some Python syntax rules in the next section. For now, we can create a simple script file named ex_1.py that has one line:

print("π≈", 355/113)

We can also use "\u03c0\u2248" instead of "π≈". The string "\N{GREEK SMALL LETTER PI}\N{ALMOST EQUAL TO}" will also work.

Once we have this file, we can have Python execute the file as follows:

MacBookPro-SLott:Chapter_1 slott$ python3 ex_1.py
π≈ 3.1415929203539825

We've provided a filename, ex_1.py, as the positional argument to the python3 program. Python reads the file and executes each line. The output that we see is the text printed to the console by the print() function.

The file is found by Python using ordinary OS rules for locating files, starting with the current working directory. This will work with any kind of filename.

If we followed the naming rules for our file—the filename is an identifier and the extension is .py—we can also use the following command to execute a Python module:

MacBookPro-SLott:Chapter_1 slott$ python3 -m ex_1
π≈ 3.1415929203539825

The -m ex_1 option forces Python to search for a module named ex_1. The file associated with this module is named ex_1.py. Python has a search path that it uses to find the requested module. Unless special arrangements are made, Python will search the local directory first, and then will search the library directories. This allows us to run our scripts and Python's built-in applications with a simple, uniform syntax. It also allows us to add our own applications and modules by modifying the PYTHONPATH environment variable.

We'll look at the search path in Chapter 12, Scripts, Modules, Packages, Libraries, and Applications. The detailed documentation for the search path is part of the site package.

Simplified syntax rules

The syntax rules for Python are defined in section 2 of the Python Language Reference manual. We'll look at the rules in detail in Chapter 3, Expressions and Output.

Python has about 20 kinds of statements. Here's a quick summary of the rules:

  • Almost all statements begin with a Python keyword such as pass, if, and def. The expression statement and the assignment statement are the exceptions.

  • Python has two kinds of statements—one-line simple statements and multiline compound statements.

  • Simple statements must be complete within a single line. An assignment statement is a simple statement. It begins with one or more user-provided identifiers and includes the = assignment symbol or an augmented variant like +=. An expression statement is also simple.

  • Compound statements use indentation to show the suite of statements embedded within the overall statement. The standard indentation is four spaces. Most developers set their editor to replace tabs with four spaces. Inconsistent use of spaces and tabs will lead to syntax errors that can be hard to see because tabs and spaces are both invisible by default. Avoiding tab characters in general makes it easier to debug problems.

  • Compound statements include class and function definitions—the body of the definition is indented. If statements and for and while loops are examples of compound statements that contain an indented suite of statements that are executed conditionally or repeatedly.

  • The ( and ) characters must match. A single statement on a logical line may span multiple physical lines until the ( and ) characters match.

In effect, Python programs consist of one-statement-one-line. The end of a line is the statement terminator. We have a few techniques for extending a statement. The most common technique is based on Python's requirement that the ( and ) characters must balance.

We can, for example, write code like this:

print(
   "Hello world",
   "π≈",
   355/113
)

Tip

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

We've spread a single logical line to four physical lines using ( and ). One consequence of this is that a simple statement that we enter at the REPL must not be indented. A leading space will cause problems because leading spaces are used to show which statements are inside a compound statement.

Another consequence of this is less direct. Python executes a script file one statement at a time from start to finish. This means that complex Python programs will have a number of definitions first, and the "main" part of the processing will generally be last.

A Python comment starts with # and ends at the end of the line. This follows the same rules as the various Linux shells. Because of the way Python documentation strings are processed by pydoc and help(), most documentation is actually presented in separate string literals at the start of a package, module, class, or function definition. We'll look at these documentation strings in Chapter 14, Fit and Finish – Unit Testing, Packaging, and Documentation. The # comment is used sparingly.

 

The Python ecosystem


The Python programming environment has two broad subject areas:

  • The language itself

  • The extension packages. We can further subdivide the extension packages into:

    • The standard library of packages

    • The Python ecosystem of yet more extension packages

When we install Python, we install the language plus several hundred extension packages in the standard library. We'll return to the standard library in Chapter 12, Scripts, Modules, Packages, Libraries, and Applications. The Python ecosystem is potentially infinite. The good news is that PyPI makes it relatively easy to locate packages.

The idea of extensibility via add-ons

Python's design includes a small core language that can be extended by importing additional features. The Language Reference Manual describes 20 statements; there are only 19 operators. The idea is that we can have a great deal of confidence that a small language is correctly implemented, complete, and consistent.

The standard library documentation contains 37 chapters, and describes hundreds of extension packages. There are a lot of features available to help us solve our unique problem. It's typical to see Python programs that import numerous packages from the standard library.

We'll see two common variations of the import statement:

  • import math

  • from math import sqrt, sin

The first version imports the entire math module and creates the module as an object in the global namespace. The various classes and function names within that module must be properly qualified with the namespace to be used. A qualified name will look similar to math.sqrt() or math.sin().

While the second version also imports the math module, it only introduces the given names into the global namespace. These names do not require qualifiers. We can use sqrt() and sin() as if they were built-in functions. The math module object, however, is not available, since it was not introduced into the global namespace.

An import happens exactly once. Python tracks the imported modules and will not import a module a second time. This allows us to freely import modules as needed without worrying about the order or other obscure dependencies among modules.

For confirmation of this one-time-only rule for imports, try the following:

>>> import this
>>> import this

The behavior the second time is different because the module has already been imported once.

Using the Python Package Index – PyPI

Many developers of Python modules will register their work with the PyPI. This is located at http://pypi.python.org/. This is the second place to look for a module that might help solve a particular problem.

The first place to look is always the standard library.

The PyPI web page has a handy search form as well as a browser that shows packages organized under nine different metadata variables. In many cases, a book or blog post may provide a direct path like this: https://pypi.python.org/pypi/Sphinx/1.3b2. This ensures that the proper version can be downloaded and installed.

There are three common ways to download and install software from the PyPI:

  • Using pip

  • Using easy_install

  • Manually

Generally, we'll use tools such as pip or easy_install for almost all of our installations. Once in a while, however, we may need to resort to a manual installation.

Some modules may involve binary extensions to Python. These are generally C-language-sources, so they must be compiled to be useful. For Windows—where C compilers are rare—it's often necessary to find an .msi installer that includes prebuilt binaries. For Mac OS X and Linux, the C source may be compiled as part of the installation process.

In the case of large, complex numeric and scientific packages—specifically, numpy and scipy—the build process can become quite complex: generally, more complex than pip or easy_install can handle. There are many additional high-performance libraries for these packages; the builds include modules in FORTRAN as well as C. In this case, a prebuilt OS-specific distribution is used; pip isn't part of the process.

Installing additional packages will require administrator privileges. Consequently, we'll show the sudo command as a reminder that this is required for Mac OS X and Linux. Windows users can simply ignore the presence of the sudo command.

Using pip to gather modules

The pip program is part of Python 3.4. It's an add-on for Python3. To use pip to install a package, we generally use a command such as the following:

prompt$ sudo pip3.4 install some-package

For Mac OS X or Linux, we need to use the sudo command so that we have administrator privileges. Windows users will leave this off.

The pip program will search PyPI for the package named some-package. The installed Python version and OS information will be used to locate the latest-and-greatest version that's appropriate for the platform. The files will be downloaded, and the Python setup.py file that comes with the package will be run automatically to install it.

For Mac OS X and Linux users, it's helpful to note that the version of Python that is required by the OS doesn't usually have pip configured. A Mac OS X user with the built-in Python 2.7 and Python 3.4 can generally use the default pip command without any problems because there won't be a version of pip configured for Python 2.

In the case where someone has Python 3.3 and Python 3.4, and has installed pip for Python 3.3, they will have to choose which version they want to work with. Using the commands pip3.3 or pip3.4 will use one of the pip commands configured for the given version of Python. The default pip command may link to whichever version was installed last-something we shouldn't guess at.

The pip program has a number of additional features to uninstall packages and track which packages have been added to the initial Python installation. The pip program can also create installable packages of your new creation.

Using easy_install to add modules

The easy_install package is also part of Python 3.4. It's a part of the setuptools package. We use easy_install like this to install a package:

prompt$ sudo easy_install-3.3 some_package

For Mac OS X or Linux, we need to use the sudo command so that we have administrator privileges. Windows users will leave this off.

The easy_install program is similar to pip—it will search PyPI for the package named some-package. The installed Python version and OS information will be used to locate a version that's appropriate for the platform. The files will be downloaded. One of these files is the setup.py script; this will be run automatically to finish the installation.

Installing modules manually

In rare cases, we may have a package that isn't in the PyPI and can't be located by pip or easy_install. In this case, we generally have a two- or three-step installation process:

  1. Download: We need to securely download the package. In many cases, we can use https or ftps so that secure sockets are used. In case we can't secure the connection, we may have to check md5 signatures on the files to be sure that our download is complete and unaltered.

  2. Unpack: If the Python packages are compressed into a single ZIP or TAR file, we need to unzip or untar the downloaded file into a temporary directory.

  3. Set up: Many Python packages designed for manual installation include a setup.py file that will do the final installation. We'll need to run a command like this:

    sudo python3 setup.py install
    

This sequence of steps, including the final command, is what is automated by pip and easy_install. We've shown the Mac OS X and Linux use of the sudo command to assure that administrator privileges are available. Windows users will simply leave this off.

The setup.py script uses Python's distutils package to define what must be installed into the Python library directory structure. The install option states what we want to do with the package we downloaded. Most of the time, we're going to install, so this is one of the most common options.

In rare exceptions, a package may consist of a single module file. There may not be a setup.py file. In this case, we will manually copy the file to our own site-packages directory.

 

Looking at other Python interpreters


This book will focus on a particular implementation of Python called the CPython. What this means is that Python—the abstract language—can be processed by a variety of concrete Python runtimes or implementations. The CPython implementation is written in portable C and can be recompiled for many operating systems.

Python can be embedded into an application. This means that a complex application can include the entire Python language as a way to write scripts that customize the given application. One example of this is the Ganglia monitoring system (http://ganglia.sourceforge.net). Python is part of the system; we can customize the behavior using Python scripts that will interact with Ganglia components. We won't be looking more deeply into these kinds of applications in this book; we'll focus on standalone implementations of Python.

There are several alternative Python implementations. In the Installing Python on Windows section in this chapter, we noted that Iron Python (http://ironpython.net) and PTVS (http://pytools.codeplex.com) are available. These provide tighter integration with the .NET framework.

There are still more implementations that we might encounter:

  • Jython: This is a version of the Python interpreter that is written in Java and runs on the Java Virtual Machine (JVM). See http://www.jython.org. This project focuses on Python 2.7.

  • PyPy: This is a version of the Python interpreter written in Python. See http://pypy.org. The circularity of "Python written in Python" is broken by the RPython translation toolchain, which creates a very sophisticated implementation of Python programs. This can provide significant performance improvements for a variety of long-running applications, such as web servers.

  • Stackless: This version of Python has a different threading model from CPython. See http://www.stackless.com. This version can provide dramatic performance improvements for multithreaded servers.

Since the Python source is readily available, it's quite easy to look for optimization opportunities. The language is relatively simple, allowing experimentation to see what effect changes in implementation may have.

 

Summary


We've looked at installing or upgrading Python so that we can work with version 3.3 or 3.4, and we've looked briefly at the minor differences between Windows, Mac OS X, and Linux. The principle difference among the OS variants is that Windows lacks Python, whereas Mac OS X and Linux generally have a version of Python already installed. There are few other differences among the operating systems.

We've looked at some basic interactions using the REPL. We looked at some simple expressions and the built-in help() subsystem.

We've looked at some ways that the import statement extends the basic capabilities of our Python runtime environment, and we've also introduced the larger Python ecosystem. We can add to our Python library using the pip (and easy_install) tool. The PyPI is the central repository for most of the Python extension modules.

In the next chapter, we'll look at Python's numeric types in detail. Python numbers form a kind of "tower" that follows the mathematical notions of integer, rational, real, and complex numbers. We'll look at the mathematical operators and some of the standard libraries for working with numbers.

We'll also look at some of the more complex data types available, including/alongside specific tuples, strings, and frozensets. These are relatively simple because they are immutable. As is the case with ordinary numbers, the values of these more complex objects don't change either.

About the Author

  • Steven F. Lott

    Steven F. Lott has been programming since the 70s, when computers were large, expensive, and rare. As a contract software developer and architect, he has worked on hundreds of projects, from very small to very large. He's been using Python to solve business problems for almost 20 years. He's currently leveraging Python to implement cloud management tools. His other titles with Packt Publishing include Python Essentials, Mastering Object-Oriented Python, Functional Python Programming, and Python for Secret Agents. Steven is currently a technomad who lives in various places on the east coast of the U.S.

    Browse publications by this author

Latest Reviews

(9 reviews total)
This book is very easy to understand for beginners. Well articulated and give very easy example as I read so far.
Except that EVERY page faded at the bottom, into a "purchase to see more" was REALLY irritating. I don't hold that against the content, though.
This is must have book. I use it occasionally for reference and for clarification.