10
The Functools Module

Functional programming considers functions to be first-class objects. We’ve seen several higher-order functions that accept functions as arguments or return functions as results. In this chapter, we’ll look at the functools library, which contains some tools to help us implement some common functional design patterns.

We’ll look at some higher-order functions. This extends the material from Chapter 5, Higher-Order Functions. We’ll continue looking at higher-order function techniques in Chapter 12, Decorator Design Techniques, as well.

We’ll look at the following functions in this module:

• @cache and @lru_cache: These decorators can be a huge performance boost for certain types of applications.

• @total_ordering: This decorator can help create rich comparison operators. Additionally, it lets us look at the more general question of object-oriented design mixed with functional programming.

• partial(): This function creates...

10.1 Function tools

We looked at a number of higher-order functions in Chapter 5, Higher-Order Functions. Those functions either accept a function as an argument or return a function (or generator expression) as a result. All those higher-order functions have an essential algorithm that is customized by injecting another function. Functions such as max(), min(), and sorted() accept a key= function to customize their behavior. Functions such as map() and filter() accept a function and an iterable and apply the given function to the argument iterable. In the case of the map() function, the results of the function are simply yielded. In the case of the filter() function, the Boolean result of the function is used to yield or reject values from an iterable source.

All the functions in Chapter 5, Higher-Order Functions, are part of the Python __builtins__ package, meaning these functions are available without the need to use the import statement. They were made ubiquitous because...

10.2 Memoizing previous results with cache

The @cache and @lru_cache decorators transform a given function into a function that might perform more quickly. LRU means Least Recently Used—a finite pool of recently used items is retained. Items not recently used are discarded to keep the pool to a bounded size. The @cache has no storage management and requires a little bit of consideration to be sure it won’t consume all available memory.

Since these are decorators, we can apply one of them to any function that might benefit from caching previous results. We can use it as follows:

from functools import lru_cache 
 
@lru_cache(128) 
def fibc(n: int) -> int: 
    if n == 0: return 0 
    if n == 1: return 1 
    return fibc(n-1) + fibc(n-2)

This is an example based on Chapter 6, Recursions and Reductions. We’ve applied the @lru_cache decorator to the naive Fibonacci number...

10.3 Defining classes with total ordering

The @total_ordering decorator is helpful for creating new class definitions that implement a rich set of comparison operators. This might apply to numeric classes that subclass numbers.Number. It may also apply to semi-numeric classes.

As an example of a semi-numeric class, consider a playing card. It has a numeric rank and a symbolic suit. The suit, for example, may not matter for some games. Like ordinary integers, cards have an ordering. We often sum the point values of each card, making them number-like. However, multiplication of cards, card × card, doesn’t really make any sense; a card isn’t quite like a number.

We can almost emulate a playing card with a NamedTuple base class as follows:

from typing import NamedTuple 
 
class Card1(NamedTuple): 
    rank: int 
    suit: str

This model suffers from a profound limitation: all comparisons between cards will include...

10.4 Applying partial arguments with partial()

The partial() function leads to something called a partial application. A partially applied function is a new function built from an old function and a subset of the required argument values. It is closely related to the concept of currying. Much of the theoretical background is not relevant here, since currying doesn’t apply directly to the way Python functions are implemented. The concept, however, can lead us to some handy simplifications.

We can look at trivial examples as follows:

>>> exp2 = partial(pow, 2)
>>> exp2(12)
4096
>>> exp2(17)-1
131071

We’ve created the function exp2(y), which is the pow(2, y) function. The partial() function binds the first positional parameter to the pow() function. When we evaluate the newly created exp2() function, we get values computed from the argument bound by the partial() function, plus the additional argument provided to the exp2() function.

The bindings...

10.5 Reducing sets of data with the reduce() function

The sum(), len(), max(), and min() functions are, in a way, all specializations of a more general algorithm expressed by the reduce() function. See Chapter 5, Higher-Order Functions for more on these functions. The reduce() function is a higher-order function that folds a binary operation into each pair of items in an iterable.

A sequence object is given as follows:

>>> d = [2, 4, 4, 4, 5, 5, 7, 9]

The expression reduce(lambda x, y: x+y, d) will fold in + operators to the list as if we were evaluating the following:

>>> from functools import reduce 
 
>>> reduce(lambda x, y: x+y, d) 
40 
>>> 2+4+4+4+5+5+7+9 
40

It can help to include () to show the effective left-to-right grouping as follows:

>>> ((((((2+4)+4)+4)+5)+5)+7)+9 
40

Python’s standard interpretation of expressions involves a left-to-right evaluation...

10.6 Handling multiple types with singledispatch

We’ll often have functions which have similar semantics but distinct implementations based on the type of data presented. We might have a function that works for either a subclass of NamedTuple, or TypedDict. The syntax for working with these objects is distinct, and we can’t use a single, generic Python function.

We have the following choices for working with data of distinct types:

• Use the match statement with a case clause for each distinct type.

• Use the @singledispatch decorator to define a number of closely-related functions. This will create the necessary type-matching match statement for us.

A small example arises when working with US postal data and spreadsheets. It’s common for a US postal ZIP code to be misinterpreted as an integer (or float) value. The town of Andover, MA, for example, has a postal code of 01810. A spreadsheet might misinterpret this as an integer, 1810, dropping the leading zero.

When...

10.7 Summary

In this chapter, we’ve looked at a number of functions in the functools module. This library module provides a number of functions that help us create sophisticated functions and classes.

We’ve looked at the @cache and @lru_cache decorators as ways to boost certain types of applications with frequent re-calculations of the same values. These two decorators are of tremendous value for certain kinds of functions that take integer or string argument values. They can reduce processing by simply implementing memoization. The @lru_cache has an upper bound on the memory it will use; this is good for a domain with an unknown size.

We looked at the @total_ordering function as a decorator to help us build objects that support rich ordering comparisons. This is at the fringe of functional programming, but is very helpful when creating new kinds of numbers.

The partial() function creates a new function with the partial application of argument values. As an alternative...

10.8 Exercises

This chapter’s exercises are based on code available from Packt Publishing on GitHub. See https://github.com/PacktPublishing/Functional-Python-Programming-3rd-Edition.

In some cases, the reader will notice that the code provided on GitHub includes partial solutions to some of the exercises. These serve as hints, allowing the reader to explore alternative solutions.

In many cases, exercises will need unit test cases to confirm they actually solve the problem. These are often identical to the unit test cases already provided in the GitHub repository. The reader should replace the book’s example function name with their own solution to confirm that it works.

10.8.1 Compare string.join() and reduce()

In the Avoiding problems with reduce() section of this chapter, we noted that we can combine a list of string values into a single string in the following two ways:

• reduce(operator.add, list_of_strings, "")

• "".join(list_of_strings...

Join our community Discord space

Join our Python Discord workspace to discuss and know more about the book: https://packt.link/dHrHU