Chapter 8. Doing Things Asynchronously
In the previous chapter, we learned how to distribute tasks using the Celery framework and parallelize computing in different machines linked by a network. Now, we are going to explore asynchronous programming, event loop, and coroutines, which are resources featured in the asyncio module available in Python Version 3.4. We are also going to learn to make use of those in combination with executors.
In this chapter, we will cover:
Understanding blocking, nonblocking, and asynchronous operations
Understanding the different approaches to task execution is extremely important to model and conceive a scalable solution. Knowing when to use asynchronous, blocking, and nonblocking operations can make an enormous difference in the response time of a system.
Understanding blocking operations
In the case of a blocking operation, we can use the example of attending a customer at a bank counter. When the customer's number is called for attendance, all the attention of the cashier is focused on this specific customer. Until the necessity of the current customer is achieved, the cashier can't attend another customer simultaneously. Now, with this in mind, imagine a bank agency with only two cashiers and an influx of 100 customers per hour; we have then a flow problem. This case illustrates the blocking of processing, when a task needs to wait for another to end, blocking the access to resources.
Tip
In the blocking of processing, the...
In order to understand the concept of event loop, we need to understand the elements that form its inner structure.
We will use the term resource descriptor
to refer to the socket descriptor as well as file descriptor.
The polling technique is implemented by different operating systems aiming to monitor the status of one or more resource descriptors. Systems implement this technique by means of functions. Polling functions form the basis of event loops. We can often find these models being referred to as
readiness notification scheme due to the fact that the polling function notifies the one interested in the event, that the resource descriptor is ready for interaction; the one interested, however, might/might not accomplish the desired operation.
In terms of Linux, for instance, we have the following polling functions:
We can define asyncio as a module that came to reboot asynchronous programming in Python. The asyncio module allows the implementation of asynchronous programming using a combination of the following elements:
Event loop: This was already defined in the previous section. The asyncio module allows an event loop per process.
Coroutines: As mentioned in the official documentation of asyncio, "A coroutine is a generator that follows certain conventions." Its most interesting feature is that it can be suspended during execution to wait for external processing (some routine in I/O) and return from the point it had stopped when the external processing is done.
Futures: The asyncio module defines its own object Future. Futures represent a processing that has still not been accomplished.
Tasks: This is a subclass of asyncio.Future to encapsulate and manage coroutines.
Beyond these mechanisms, asyncio provides a series of other features for the developing of applications, such as transports...
In this chapter, we have learned about asynchronous, blocking, and nonblocking programming. We have made use of some basic mechanisms of asyncio in order to see the nuts and bolts of this mechanism's behavior in some situations.
The asyncio module is an attempt to reboot the support to asynchronous programming in Python. Guido Van Rossum was extremely successful in exploring alternatives and thinking of something that could be used as a basis to these alternatives offering a clear API. The yield from syntax was born to enhance the expressivity of some programs that use coroutines, relieving the burden on the developer of writing callbacks to treat the ending of events, although it is possible to use callbacks. The asyncio module, beyond other advantages, has the capacity of integrating with other applications, as in the Tornado web server, for instance, that already has a support branch to event loop in asyncio.
We come to the end of this book, which was indeed challenging to write...
