Expert Python Programming - Third Edition

4.3 (6 reviews total)
By Michał Jaworski , Tarek Ziadé
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Current Status of Python

About this book

Python is a dynamic programming language that's used in a wide range of domains thanks to its simple yet powerful nature. Although writing Python code is easy, making it readable, reusable, and easy to maintain is challenging. Complete with best practices, useful tools, and standards implemented by professional Python developers, the third edition of Expert Python Programming will help you overcome this challenge.

The book will start by taking you through the new features in Python 3.7. You'll then learn the advanced components of Python syntax, in addition to understanding how to apply concepts of various programming paradigms, including object-oriented programming, functional programming, and event-driven programming. This book will also guide you through learning the best naming practices, writing your own distributable Python packages, and getting up to speed with automated ways of deploying your software on remote servers. You’ll discover how to create useful Python extensions with C, C++, Cython, and CFFI. Furthermore, studying about code management tools, writing clear documentation, and exploring test-driven development will help you write clean code.

By the end of the book, you will have become an expert in writing efficient and maintainable Python code.

Publication date:
April 2019
Publisher
Packt
Pages
646
ISBN
9781789808896

 

Chapter 1. Current Status of Python

Python is amazing.

For a very long time, one of the most important virtues of Python was interoperability. No matter what operating system you or your customers were using, if a Python interpreter was available for that system, your software that was written in Python would work there. And, most importantly, your software would work the same way. However, that's not uncommon anymore. Modern languages such as Ruby and Java provide similar interoperability capabilities. But, interoperability isn't the most important quality of programming language nowadays. With the advent of cloud computing, web-based applications, and reliable virtualization software, it isn't that important to have a programming language that works the same no matter the operating system. What is still important is the tools that allow programmers to efficiently write reliable and maintainable software. Fortunately, Python is still one of the languages that allows programmers the most efficiency, and is definitely a smart choice for a company's primary development language.

Python stays relevant for so long because it is constantly evolving. This book is focused on the latest Python 3.7 version, and all code examples are written in this version of the language unless another version is explicitly mentioned. Because Python has a very long history, and there are still programmers using Python 2 on a daily basis, this book starts with a chapter that describes the current status quo of Python 3. In this chapter, you'll find how and why Python changes, and will learn how to write software that is compatible with both the historic and latest versions of Python.

In this chapter, we will cover the following topics:

  • Where are we now and where we are going to?
  • Why and how Python changes
  • Being up-to-date with changes to PEP documentation
  • Python 3 adoption at the time of writing this book
  • The main difference between Python 3 and Python 2
  • Not only CPython
  • Useful resources
 

Technical requirements


You can download the latest version of Python from https://www.python.org/downloads/ for this chapter.

Alternative Python interpreter implementations can be found at the following sites:

The code files for this chapter can be found at https://github.com/PacktPublishing/Expert-Python-Programming-Third-Edition/tree/master/chapter1.

 

Where are we now and where we are going to?


Python history starts somewhere in the late 1980s, but its 1.0 release date was in the year 1994. So, it isn't a young language. There could be a whole timeline of major Python releases mentioned here, but what really matters is a single date: Python 3.0—December 3, 2008.

 

 

At the time of writing, almost ten years have passed since the first Python 3 release. It is also seven years since the creation of PEP 404—the official document that un-released Python 2.8 and officially closed the 2.x branch. Although a lot of time has passed, there is a specific dichotomy in the Python community—while the language is developing very fast, there is a large group of its users that do not want to move forward with it.

 

Why and how Python changes


The answer is simple—Python changes because there is such a need. The competition does not sleep. Every few months, a new language pops out, out of nowhere, claiming to solve every problem of all its predecessors. Most projects like these lose the developers' attention very shortly, and their popularity is often driven by sudden hype.

This is a sign of some bigger problem. People design new languages because they find that existing ones do not solve their problems in the best way possible. It would be silly to not recognize such a need. Also, more and more widespread usage of Python shows that it could, and should, be improved on in many places.

Many improvements in Python are driven by the needs of particular fields where it is being used. The most significant one is web development. Thanks to the ever-increasing demand for speed and performance in this area, we've seen that ways to deal with concurrency in Python have been drastically improved over the time.

Other changes are simply caused by the age and maturity of the Python project. Throughout the years, it collected some of the clutter in the form of disorganized and redundant standard library modules, or some bad design decisions. First, the Python 3 release aimed to bring with it a major cleanup and refreshment to the language. Unfortunately, time showed that this plan backfired a bit. For a long time, Python 3 was treated by many developers only like a curiosity. Hopefully, this is changing.

 

Being up-to-date with changes ­by following PEP documents


The Python community has a well-established way of dealing with changes. While speculative Python language ideas are mostly discussed on specific mailing lists ([email protected]), nothing major ever gets changed without the existence of a new document, called a Python Enhancement Proposal (PEP).

 

 

It is a formalized document that describes, in detail, the proposal of change to be made in Python. It is also the starting point for the community discussion. The whole purpose, format, and workflow around these documents is also standardized in the form of a PEP—precisely the PEP 1 document (http://www.python.org/dev/peps/pep-0001).

PEP documentation is very important for Python, and, depending on the topic, they serve different purposes:

  • Informing: They summarize the information needed by core Python developers, and notify about Python release schedules
  • Standardizing: They provide code style, documentation, or other guidelines
  • Designing: They describe the proposed features

A list of all proposed PEPs are available in a living PEP 0 document (https://www.python.org/dev/peps/). Since they are easily accessible in one place, and the actual URL is also very easy to guess, they are usually referred to by the number in the book.

The PEP 0 document is a great source of information for those who are wondering what direction Python language is heading in, but do not have time to track every discussion on Python mailing lists. It shows which documents were already accepted but not yet implemented, and also which are still under consideration.

PEPs also serve additional purposes. Very often, people ask questions like the following:

  • Why does feature A work that way?
  • Why does Python not have feature B?

In most such cases, the extensive answer is already available in specific PEP documents where such a feature was already mentioned. There is a lot of PEP documentation describing Python language features that were proposed but not accepted. This documentation is left as a historical reference.

 

Python 3 adoption at the time of writing this book


So, thanks to new, exciting features, is Python 3 well adopted among its community? It's hard to say. The once-popular page, Python 3 Wall of Superpowers (https://python3wos.appspot.com), that tracked the compatibility of the most popular packages with the Python 3 branch was, at the beginning, named Python 3 Wall of Shame.

 

 

The site is no longer maintained, but in the list from the last time it was updated, on April 22, 2018, it shows that exactly 191 from 200 of the most popular Python packages at that time were compatible within Python 3. So, we can see that Python 3 seems to be finally well-adopted in the community of open source Python programmers. Still, this does not mean that all teams building their applications are finally using Python 3. At least, since most of the popular Python packages are available in Python 3, the popular excuse packages that we use have not been ported yet is no longer valid.

The main reason for such a situation is that porting the existing application from Python 2 to Python 3 is always a challenge. There are tools such as 2to3 that can perform automated code translation, but they do not assure that the result will be 100% correct. Also, such translated code may not perform as well as in its original form without manual adjustments. Moving existing complex code bases to Python 3 might involve tremendous effort, and a cost that some organizations may not be able to afford. Fortunately, such costs can be split over time. Some good software architecture design methodologies, such as service-oriented architecture or microservices, can help to achieve this goal gradually. New project components (services or microservices) can be written using the new technology, and existing ones can be ported one at a time.

In the long run, moving to Python 3 can have only beneficial effects on a project. According to PEP 404, there won't be another 2.8 release in the 2.x branch of Python, and the official end-of-life for Python 2 is scheduled for 2020. Until that time, we can expect only patch version updates for major security issues, but nothing more. Also, there may be a time in the future when all major projects, such as Django, Flask, and NumPy will drop any 2.x compatibility and will be available only in Python 3. Django has already made that step, and since version 2.0.0 was released, it no longer supports Python 2.7.

My personal opinion on this topic can be considered controversial. I think that the best incentive for the community would be to completely drop Python 2 support when creating new packages. This, of course, limits a range of such software, but may be the only right way to change the way of thinking in those who insist on sticking to Python 2.x.

We'll take a look at the main differences between Python 3 and Python 2 in the next section.

 

 

 

The main differences between Python 3 and Python 2


It has already been stated that Python 3 breaks backward compatibility with Python 2 on a syntax level. Still, it is not a complete redesign. Also, it does not cause every Python module written for some 2.x release to stop working under Python 3. It is possible to write completely cross-compatible code that will run on both major releases without additional tools or techniques, but usually it is possible only for simple applications.

Why should I care?

Despite my personal opinion on Python 2 compatibility that I exposed earlier in this chapter, it is impossible to simply forget about it at this time. There are still some useful packages that are really worth using, but are not likely to be ported in the very near future.

Also, sometimes, we may be constrained by the organization we work in. The existing legacy code may be so complex that porting it is not economically feasible. So, even if we decide to move on and live only in the Python 3 world from now on, it will be impossible to live completely without Python 2 for some time.

Nowadays, it is very hard to call yourself a professional developer without giving something back to the community. So, helping the open source developers add Python 3 compatibility to the existing packages is a good way to pay off the moral debt incurred by using them. This, of course, cannot be done without knowing the differences between Python 2 and Python 3. By the way, this is also a great exercise for those new to Python 3.

The main syntax differences and common pitfalls

The Python documentation is the best reference for differences between every Python release. However, for your convenience, this section summarizes the most important ones. This does not change the fact that the documentation is mandatory reading for those not familiar with Python 3 yet (see https://docs.python.org/3.0/whatsnew/3.0.html).

 

 

The breaking changes that were introduced by Python 3 can be generally divided into three groups:

  • Syntax changes, where some syntax elements were removed/changed and other elements were added
  • Changes in the standard library
  • Changes in datatypes and collections

Syntax changes

Syntax changes that make it difficult for the existing code to run are the easiest to spot—they will cause the code to not run at all. The Python 3 code that uses new syntax elements will fail to run on Python 2 and vice versa. The elements that were removed from official syntax will make Python 2 code visibly incompatible with Python 3. Any attempt to run the code that has such issues will immediately cause the interpreter to fail, raising a SyntaxError exception. Here is an example of the broken script that has exactly two statements, of which none will be executed due to the syntax error:

print("hello world")
print "goodbye python2"

Its actual result when run on Python 3 is as follows:

$ python3 script.py
  File "script.py", line 2
    print "goodbye python2"
                          ^
  SyntaxError: Missing parentheses in call to 'print'

When it comes to new elements of Python 3 syntax, the total list of differences is a bit long, and any new Python 3.x release may add new elements of syntax that will raise such errors on earlier releases of Python (even on the same 3.x branch). The most important of them are covered in Chapter 2, Modern Python Development Environments, and Chapter 3, Modern Syntax Elements – Below the Class Level, so there is no need to list all of them here.

The list of things that used to work in Python 2 that will cause syntax or functional errors in Python 3 is shorter. Here are the most important backwards incompatible changes:

  • print is no longer a statement, but a function, so the parenthesis is now obligatory.
  • Catching exceptions changed from except exc, var to except exc as var.
  • The <> comparison operator has been removed in favor of !=.
  • from module import * (https://docs.python.org/3.0/reference/simple_stmts.html#import) is now allowed only on module level, and no longer inside the functions.
  • from .[module] import name is now the only accepted syntax for relative imports. All imports not starting with a dot character are interpreted as absolute imports.
  • The sorted() function and the list's sort() method no longer accept the cmp argument. The key argument should be used instead.
  • Division expressions on integers such as one half return floats. The truncating behavior is achieved through the // operator like 1//2. The good thing is that this can be used with floats too, so 5.0//2.0 == 2.0.

Changes in the standard library

Breaking changes in the standard library are the second easiest to catch after syntax changes. Each subsequent version of Python adds, deprecates, improves, or completely removes standard library modules. Such a process was also common in the older branches of Python (1.x and 2.x), so it does not come as a shock in Python 3. In most cases, depending on the module that was removed or reorganized (such as urlparse being moved to urllib.parse), it will raise exceptions on the import time just after it is interpreted. This makes such issues so easy to catch. In order to be sure that all such issues are covered, full test code coverage is essential. In some cases (for example, when using lazily loaded modules), the issues that are usually noticed at import time will not appear before some modules are used in code as function calls. This is why it is so important to make sure that every line of code is actually executed during tests suite.

Note

Lazily loaded modules A lazy loaded module is a module that is not loaded on import time. In Python, the import statements can be included inside functions, so an import will happen on function call and not on import time. In some cases, such loading of modules may be a reasonable choice, but in most cases, it is a workaround for poorly designed module structure (for example, to avoid circular imports). It is considered bad code smell and should be generally avoided. There is no justifiable reason to lazily load standard library modules. In well-structured code, all imports should be grouped at the top of module.

 

Changes in data types and collections and string literals

Changes in how Python represents datatypes and collections require the most effort when the developer tries to maintain compatibility or simply ports existing code to Python 3. While incompatible syntax or standard library changes are easily noticeable and often easy to fix, changes in collections and types are either non-obvious or require a lot of repetitive work. The list of such changes is long and the official documentation is the best reference.

Still, this section must cover the change in how string literals are treated in Python 3, because it seems to be the most controversial and discussed change in Python 3, despite being a very good move that makes things more explicit.

All string literals are now Unicode, and bytestring literals require b or B prefix. For Python 3.0 and 3.1, the old Unicode u prefix (like u"foo") is illegal and will raise a syntax error. Dropping off that prefix was the main reason for most of the controversies. It made it really hard to create code compatible with different branches of Python—Python in version 2.x relied on these prefixes in order to create Unicode literals. This prefix was brought back in Python 3.3 to ease the integration process, although it now lacks any syntactic meaning.

The popular tools and techniques used for maintaining cross-version compatibility

Maintaining compatibility between versions of Python is a challenge. It may add a lot of additional work depending on the size of the project, but is definitely doable and worth doing. For packages that are meant to be reused in many environments it is absolutely a must-have. Open source packages without well-defined and tested compatibility bounds are very unlikely to become popular, but closed third-party code that never leaves the company network can also greatly benefit from being tested in different environments.

It should be noted here that, while this part focuses mainly on compatibility between various versions of Python, these approaches apply for maintaining compatibility with external dependencies such as different package versions, binary libraries, systems, or external services.

 

 

The whole process can be divided into three main areas, ordered by their importance:

  • Defining and documenting target compatibility bounds and how they will be managed
  • Testing in every environment and with every dependency version declared as compatible
  • Implementing actual compatibility code

Declaration of what is considered compatible is the most important part of the whole process because it gives your code users (developers) the ability to have expectations and make assumptions on how it works and how it can change in the future. Our code can be used as a dependency in different projects that may also strive to manage compatibility, so the ability to reason how it behaves is crucial.

While this book tries to always give a few choices and not to give absolute recommendations on specific options, here is one of the few exceptions. The best way to define how compatibility may change in the future is by using proper approach to versioning numbers using Semantic Versioning (semver) (http://semver.org/). It describes a broadly accepted standard for marking scope of changes in code by the version specifier, consisting only of three numbers. It also gives some advice on how to handle deprecation policies. Here is an excerpt from its summary (licensed under Creative Commons - CC BY 3.0):

Given a version number MAJOR.MINOR.PATCH, increment:

  1. MAJOR version when you make incompatible API changes,
  2. MINOR version when you add functionality in a backwards-compatible manner, and
  3. PATCH version when you make backward-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

 

 

When it comes to testing the sad truth, that is, to be sure that code is compatible with every declared dependency version and in every environment (here Python version), it must be tested in every combination of these. This, of course, may not be possible when the project has a lot of dependencies, because the number of combinations grows rapidly with every new dependency version. So, typically some trade-off needs to be made so that running full compatibility tests does not need to take ages. The selection of tools that help testing in so-called matrixes is presented in Chapter 12, Test-Driven Development, which discusses testing in general.

Note

The benefit of using projects that follow semver is that usually what needs to be tested are only major releases, because minor and patch releases are guaranteed to not include backwards incompatible changes. This is, of course, only true if such projects can be trusted to not break such a contract. Unfortunately, mistakes happen to everyone, and backwards incompatible changes happen in a lot of projects, even on patch versions. Still, since semver declares strict compatibility on minor and patch versions, breaking it is considered a bug, so it may be fixed in a patch release.

The implementation of the compatibility layer is the last, and also the least important, step of the process if the bounds of that compatibility are well-defined and rigorously tested. Still, there are some tools and techniques that every programmer interested in such a topic should know.

The most basic is Python's __future__ module. It backports some features from newer Python releases back into the older ones and takes the form of an import statement:

from __future__ import <feature>

Features provided by the future statements are syntax-related elements that cannot be easily handled by different means. This statement affects only the module where it was used. Here is an example of a Python 2.7 interactive session that brings Unicode literals from Python 3.0:

Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> type("foo")  # old literals
<type 'str'>
>>> from __future__ import unicode_literals
>>> type("foo")  # now is Unicode
<type 'unicode'>

 

 

 

 

 

 

 

Here is a list of all the available __future__ statement options that developers concerned with two-thirds compatibility should know:

  • division: This adds a Python 3 division operator (PEP 238)
  • absolute_import: This makes every form of an import statement not starting from dot character be interpreted as absolute imports (PEP 328)
  • print_function: This changes a print statement into a function call so that parentheses around print become mandatory (PEP 3112)
  • unicode_literals: This makes every string literal be interpreted as Unicode literals (PEP 3112)

A list of the __future__ statement options is very short, and it covers only a few syntax features. The other things that have changed, such as the metaclass syntax (which is an advanced feature that's covered in Chapter 5, Elements of Metaprogramming), are a lot harder to maintain. Reliable handling of multiple standard library reorganizations also cannot be solved by the future statements. Fortunately, there are some tools that aim to provide a consistent layer of ready-to-use compatibility code. The most well-known of these is Six (https://pypi.python.org/pypi/six/), which provides a whole common two-thirds compatibility boilerplate as a single module. The other promising, but slightly less popular, tool is the future module (http://python-future.org/).

In some situations, developers may not want to include additional dependencies in some small packages. A common practice is the additional module that gathers all the compatibility code, usually named compat.py. Here is an example of such compat modules taken from the python-gmaps project (https://github.com/swistakm/python-gmaps):

# -*- coding: utf-8 -*-
"""This module provides compatibility layer for
selected things that have changed across Python versions.
"""
import sys 
 
if sys.version_info < (3, 0, 0): 
    import urlparse  # noqa 
 
    def is_string(s):
        """Return True if given value is considered string"""
        return isinstance(s, basestring) 
 
else:
    # note: urlparse was moved to urllib.parse in Python 3
    from urllib import parse as urlparse  # noqa 
 
    def is_string(s):
        """Return True if given value is considered string"""
        return isinstance(s, str)

Such compat.py modules are popular, even in projects that depend on Six for two-thirds compatibility, because it is a very convenient way to store code that handles compatibility with different versions of packages being used as dependencies.

In the next section, we'll take a look at what CPython is.

 

Not only CPython


The reference Python interpreter implementation is called CPython and, as its name suggests, it is written entirely in the C language. It was always C and probably will be still for a very long time. That's the implementation that most Python programmers choose because it is always up to date with the language specification and is the interpreter that most libraries are tested on. But, besides C, Python interpreter was written in a few other languages. Also, there are some modified versions of CPython interpreter available under different names and tailored exactly for some niche applications. Most of them are a few milestones behind CPython, but provide a great opportunity to use and promote the language in a specific environment.

In this section, we will discuss some of the most prominent and interesting alternative Python implementations. 

Why should I care?

There are plenty of alternative Python implementations available. The Python wiki page on that topic (https://wiki.python.org/moin/PythonImplementations) features dozens of different language variants, dialects, or implementations of Python interpreter built with something other than C. Some of them implement only a subset of the core language syntax, features, and built-in extensions, but there are at least a few that are almost fully compatible with CPython. The most important thing to know is that, while some of them are just toy projects or experiments, most of them were created to solve some real problems – problems that were either impossible to solve with CPython or required too much of the developer's effort.

 

 

Examples of such problems are as follows:

  • Running Python code on embedded systems
  • Integration with code written for runtime frameworks, such as Java or .NET, or in different languages
  • Running Python code in web browsers

The following sections provide a short description of, subjectively, the most popular and up-to-date choices that are currently available for Python programmers.

Stackless Python

Stackless Python advertises itself as an enhanced version of Python. Stackless is named so because it avoids depending on the C call stack for its own stack. It is, in fact, a modified CPython code that also adds some new features that were missing from the core Python implementation at the time Stackless was created. The most important of these are microthreads, which are managed by the interpreter as cheap and lightweight alternatives to ordinary threads, that must depend on system kernel context switching and task scheduling.

The latest available versions are 2.7.15 and 3.6.6 and implement 2.7 and 3.6 versions of Python, respectively. All the additional features provided by Stackless are exposed as a framework within this distribution through the built-in stackless module.

Stackless isn't the most popular alternative implementation of Python, but it is worth knowing, because some of the ideas that were introduced in it had a strong impact on the language community. The core switching functionality was extracted from Stackless and published as an independent package named greenlet, which is now the basis for many useful libraries and frameworks. Also, most of its features were re-implemented in PyPy—another Python implementation that will be featured later. The official online documentation of Stackless Python can be found at https://stackless.readthedocs.io and the project wiki can be found at https://github.com/stackless-dev/stackless.

 

 

Jython

Jython is a Java implementation of the language. It compiles the code into Java byte code, and allows the developers to seamlessly use Java classes within their Python modules. Jython allows people to use Python as the top-level scripting language on complex application systems, for example, J2EE. It also brings Java applications into the Python world. Making Apache Jackrabbit (which is a document repository API based on JCR; see http://jackrabbit.apache.org) available in a Python program is a good example of what Jython allows.

The main differences of Jython compared to the CPython implementation are as follows:

  • True Java's garbage collection instead of reference counting
  • Lack of global interpreter lock (GIL) allows better utilization of multiple cores in multi-threaded applications

The main weakness of this implementation of the language is the lack of support for C Python Extension APIs, so no Python extensions written in C will work with Jython.

The latest available version of Jython is Jython 2.7, and this corresponds to the 2.7 version of the language. It is advertised as implementing nearly all of the core Python standard library and using the same regression test suite. Unfortunately, Jython 3.x was never released, and the project can be now safely considered dead. However, Jython is still worth mentioning, even if it is not developed anymore, because it was very unique implementation at the time and had meaningful impact on other alternative Python implementations.

The official project page can be found at http://www.jython.org.

IronPython

IronPython brings Python into the .NET Framework. The project is supported by Microsoft, where IronPython's lead developers work. It is quite an important implementation for the promotion of a language. Besides Java, the .NET community is one of the biggest developer communities out there. It is also worth noting that Microsoft provides a set of free development tools that turn Visual Studio into a full-fledged Python IDE. This is distributed as Visual Studio plugins named Python Tools for Visual Studio (PVTS), and is available as open source code on GitHub (http://microsoft.github.io/PTVS).

 

 

The latest stable release is 2.7.8, and it is compatible with Python 2.7. Unlike Jython, we can observe active development on both 2.x and 3.x branches of the interpreter, although Python 3 support still hasn't been officially released yet. Despite the fact that .NET runs primarily on Microsoft Windows, it is also possible to run IronPython on macOS and Linux. This is thanks to Mono, a cross platform, open source .NET implementation.

The main differences and advantages of IronPython compared to CPython are as follows:

  • Similar to Jython, the lack of global interpreter lock (GIL) allows for better utilization of multiple cores in multi-threaded applications
  • Code written in C# and other .NET languages can be easily integrated in IronPython and vice versa
  • It can be run in all major web browsers through Silverlight (although Microsoft will stop supporting Silverlight in 2021)

When speaking about weaknesses, IronPython seems very similar to Jython, because it does not support the Python/C Extension APIs. This is important for developers who would like to use packages such as NumPy, which are largely based on C extensions. There were a few community attempts to bring the Python/C Extensions API support to IronPython, or at least to provide compatibility for the NumPy package, but unfortunately no project had notable success in that area.

You can learn more about IronPython from its official project page at http://ironpython.net/.

PyPy

PyPy is probably the most exciting alternative implementation of Python, as its goal is to rewrite Python in Python. PyPy in the Python interpreter is written in Python. We have a C code layer carrying out the nuts-and-bolts work for CPython. But in PyPy, this C code layer is rewritten in pure Python.

This means that you can change the interpreter's behavior during execution time, and implement code patterns that couldn't be easily done in CPython.

PyPy is currently fully compatible with Python version 2.7.13, while the latest PyPy3 is compatible with Python version 3.5.3.

 

 

In the past, PyPy was mostly interesting for theoretical reasons, and it interested those who enjoyed going deep into the details of the language. It was not generally used in production, but this has changed through the years. Nowadays, many benchmarks show that, surprisingly, PyPy is often way faster than the CPython implementation. This project has its own benchmarking site that tracks performance of each version measured using dozens of different benchmarks (refer to http://speed.pypy.org/). It clearly shows that PyPy with JIT enabled is usually at least few times faster than CPython. This and other features of PyPy makes more and more developers decide to use PyPy in their production environments.

The main differences of PyPy compared to CPython implementation are as follows:

  • Garbage collection used instead of reference counting
  • Integrated tracing JIT compiler that allows impressive improvements in performance
  • Application-level Stackless features borrowed from Stackless Python

Like almost every other alternative Python implementation, PyPy lacks the full official support of C's Python Extension API. Still, it at least provides some sort of support for C extensions through its CPyExt subsystem, although it is poorly documented and still not feature complete. Also, there is an ongoing effort within the community in porting NumPy to PyPy because it is the most requested feature.

The official PyPy project page can be found at http://pypy.org.

MicroPython

MicroPython is one of the youngest alternative implementations on that list, as its first official version was released on May 3, 2014. It is also one of the most interesting implementations. MicroPython is a Python interpreter that was optimized for use on microcontrollers and in very constrained environments. Its small size and multiple optimizations allow it to run in just 256 kilobytes of code space and with just 16 kilobytes of RAM.

The main reference devices that you can test this interpreter on are BBC's micro:bit devices and pyboards, which are simple-to-use microcontroller development boards, that are targeted at teaching programming and electronics.

 

 

MicroPython is written in C99 (it's C language standard) and can be built for many hardware architectures, including x86, x86-64, ARM, ARM Thumb, and Xtensa. It is based on Python 3, but due to many syntax differences, it's impossible to say that it is fully compatible with any Python 3.x release. It is certainly a dialect of Python 3, with print() functions, async/await keywords, and many other Python 3 features, but you can't expect that your favorite Python 3 libraries will work properly under that interpreter out of the box.

You can learn more about MicroPython from its official project page at https://micropython.org.

 

Useful resources


The best way to know the status of Python is to stay informed about what's new and to constantly read Python-related resources. The web is full of such resources. The most important and obvious ones were already mentioned earlier, but here they are repeated to keep this list consistent:

  • Python documentation
  • Python Package Index (PyPI)
  • PEP 0 – Index of Python Enhancement Proposals (PEPs)

The other resources, such as books and tutorials, are useful, but often get outdated very fast. What does not get outdated are the resources that are actively curated by the community or released periodically. The few that are worth recommending are as follows:

  • Awesome Python (https://github.com/vinta/awesome-python) includes a curated list of popular packages and frameworks.
  • r/Python (https://www.reddit.com/r/Python/) is a Python subreddit where you can find news and interesting questions about Python posted by many members of Python community every day. 
  • Python Weekly (http://www.pythonweekly.com/) is a popular newsletter that delivers to its subscriber's dozens of new interesting Python packages and resources every week.
  • Pycoder's Weekly (https://pycoders.com) is another popular weekly newsletter with a digest of new packages and interesting articles. Due to its nature, the content of that newsletter often overlaps with Python Weekly, but sometimes you can find something unique that hasn't been posted elsewhere.  

 

 

These resources will provide you with tons of additional reading for countless hours.

 

Summary


This chapter concentrated on the current status of Python and the process of change that was visible throughout the history of that language. We started with a discussion of how and why Python changes and described what the main results of that process are, including the differences between Python 2 and 3. We've learned how to reliably deal with those changes and learned some useful techniques that allow us to provide code that is compatible with various versions of Python and different versions of its libraries.

Then, we took a different look at the idea of changes in programming language. We've reviewed some of the popular alternative Python interpreters and discussed their main differences compared to default CPython implementation.

In the next chapter, we will describe modern ways of setting up repeatable and consistent development environments for Python programmers and discuss the two popular tools for environment isolation: virtualenv-type environments and Docker containers.

About the Authors

  • Michał Jaworski

    Michał Jaworski has 10 years' of professional experience in Python. He has been in various roles at different companies, from an ordinary full-stack developer, through software architect, to VP of engineering in a fast-paced start-up company. He is currently a senior backend engineer at Showpad. He is highly experienced in designing high-performance distributed services. He is also an active contributor to many open source Python projects.

    Browse publications by this author
  • Tarek Ziadé

    Tarek Ziadé is a Python developer located in the countryside near Dijon, France. He works at Mozilla in the services team. He founded a French Python user group called Afpy, and has written several books about Python in French and English. When he is not hacking on his computer or hanging out with his family, he's spending time between his two passions, running and playing the trumpet.

    You can visit his personal blog (Fetchez le Python) and follow him on Twitter (tarek_ziade).

    Browse publications by this author

Latest Reviews

(6 reviews total)
I wanted to improve my python skills but it is too much about package setup and not about "expert Python programming". Package setup is important but not for me.
Testo ottimo ed accurato, in qualche punto fin troppo dettagliato. Purtroppo alcuni argomenti sono fuori dalla mia portata
El documento es una buena referencia bibliográfica.

Recommended For You

Python Machine Learning - Third Edition

Applied machine learning with a solid foundation in theory. Revised and expanded for TensorFlow 2, GANs, and reinforcement learning.

By Sebastian Raschka and 1 more
Hands-On Docker for Microservices with Python

A step-by-step guide to building microservices using Python and Docker, along with managing and orchestrating them with Kubernetes

By Jaime Buelta
The Python Workshop

Cut through the noise and get real results with a step-by-step approach to learning Python 3.X programming

By Andrew Bird and 4 more
Python GUI Programming Cookbook - Third Edition

Over 90 recipes to help you develop widgets, forms, layouts, charts, and much more using the latest features of Python 3

By Burkhard Meier