4
Working with Collections

Python offers a number of functions that process whole collections. They can be applied to sequences (lists or tuples), sets, mappings, and iterable results of generator expressions. We’ll look at Python’s collection-processing features from a functional programming viewpoint.

We’ll start out by looking at iterables and some simple functions that work with iterables. We’ll look at some design patterns to handle iterables and sequences with recursive functions as well as explicit for statements. We’ll look at how we can apply a scalar function to a collection of data with a generator expression.

In this chapter, we’ll show you examples of how to use the following functions with collections:

• any() and all()

• len(), sum(), and some higher-order statistical processing related to these functions

• zip() and some related techniques to structure and flatten lists of data

• sorted() and reversed() to impose an ordering on...

4.1 An overview of function varieties

We need to distinguish between two broad species of functions, as follows:

• Scalar functions: These apply to individual values and compute an individual result. Functions such as abs(), pow(), and the entire math module are examples of scalar functions.

• Collection functions: These work with iterable collections.

We can further subdivide these collection functions into three subspecies:

• Reduction: This uses a function to fold values in the collection together, resulting in a single final value. For example, if we fold + operations into a sequence of integers, this will compute the sum. This can be also be called an aggregate function, as it produces a single aggregate value for an input collection. Functions like sum() and len() are examples of reducing a collection to a single value.

• Mapping: This applies a scalar function to each individual item of a collection; the result is a collection of the same size. The built-in map() function does this...

4.2 Working with iterables

As noted in the previous chapters, Python’s for statement works with iterables, including Python’s rich variety of collections. When working with materialized collections such as tuples, lists, maps, and sets, the for statement involves the explicit management of state.

While this strays from purely functional programming, it reflects a necessary optimization for Python. The state management is localized to an iterator object that’s created as a part of the for statement evaluation; we can leverage this feature without straying too far from pure, functional programming. If, for example, we use the for statement’s variable outside the indented body of the statement, we’ve strayed from purely functional programming by leveraging this state control variable.

We’ll return to this in Chapter 6, Recursions and Reductions. It’s an important topic, and we’ll just scratch the surface in this section with a...

4.3 Using any() and all() as reductions

The any() and all() functions provide boolean reduction capabilities. Both functions reduce a collection of values to a single True or False. The all() function ensures that all items have a true value; the any() function ensures that at least one item has a true value. In both cases, these functions rely on the Pythonic concept of ”truish”, or truthy: values for which the built-in bool() function returns true. Generally, ”falsish” values include False and None, as well as zero, an empty string, and empty collections. Non-false values are true.

These functions are closely related to a universal quantifier and an existential quantifier used to express mathematical logic. We may, for example, want to assert that all elements in a given collection have a property. One formalism for this could look like the following:

We read this as for all x in S, the function, Prime(x), is true. We’ve used the universal quantifier...

4.4 Using len() and sum() on collections

The len() and sum() functions provide two simple reductions—a count of the elements and the sum of the elements in a sequence. These two functions are mathematically similar, but their Python implementation is quite different.

Mathematically, we can observe this cool parallelism:

• The len() function returns the sum of ones for each value in a collection, X: ∑ _x∈X1 = ∑ _x∈Xx⁰.

• The sum() function returns the sum of each value in a collection, X: ∑ _x∈Xx = ∑ _x∈Xx¹.

The sum() function works for any iterable. The len() function doesn’t apply to iterables; it only applies to sequences. This little asymmetry in the implementation of these functions is a little awkward around the edges of statistical algorithms.

As noted above, for empty sequences, both of these functions return a proper additive identity value of zero:

>>> sum(()) 
0 
>>> len(()) ...

4.5 Using zip() to structure and flatten sequences

The zip() function interleaves values from several iterators or sequences. It will create n tuples from the values in each of the n input iterables or sequences. We used it in the previous section to interleave data points from two sets of samples, creating two-tuples.

The zip() function is a generator. It does not materialize a resulting collection.

The following is an example of code that shows what the zip() function does:

>>> xi = [1.47, 1.50, 1.52, 1.55, 1.57, 1.60, 1.63, 1.65, 
... 1.68, 1.70, 1.73, 1.75, 1.78, 1.80, 1.83,] 
>>> yi = [52.21, 53.12, 54.48, 55.84, 57.20, 58.57, 59.93, 61.29, 
... 63.11, 64.47, 66.28, 68.10, 69.92, 72.19, 74.46,] 
 
>>> zip(xi, yi) 
<zip object at ...> 
 
>>> pairs = list(zip(xi, yi)) 
>>> pairs[:3] 
[(1.47, 52.21), (1.5, 53.12), (1.52, 54.48)] 
>>> pairs[-3:] 
[(1.78, 69...

4.6 Using sorted() and reversed() to change the order

Python’s sorted() function produces a new list by rearranging the order of items in a list. This is similar to the way the list.sort() method changes the order of list.

Here’s the important distinction between sorted(aList) and aList.sort():

• The aList.sort() method modifies the aList object. It can only be meaningfully applied to a list object.

• The sorted(aList) function creates a new list from an existing collection of items. The source object is not changed. Further, a variety of collections can be sorted. A set or the keys of a dict can be put into order.

There are times when we need a sequence reversed. Python offers us two approaches to this: the reversed() function, and slices with reversed indices.

For example, consider performing a base conversion to hexadecimal or binary. The following code is a simple conversion function:

from collections.abc import Iterator 
 
def digits(x: int, base: int) ->...

4.7 Using enumerate() to include a sequence number

Python offers the enumerate() function to apply index information to values in a sequence or iterable. It performs a specialized kind of wrap that can be used as part of an unwrap(process(wrap(data))) design pattern.

It looks like the following code snippet:

>>> xi[:3] 
[1.47, 1.5, 1.52] 
>>> len(xi) 
15 
 
>>> id_values = list(enumerate(xi)) 
>>> id_values[:3] 
[(0, 1.47), (1, 1.5), (2, 1.52)] 
>>> len(id_values) 
15

The enumerate() function transformed each input item into a pair with a sequence number and the original item. It’s similar to the following:

zip(range(len(source)), source)

An important feature of enumerate() is that the result is an iterable and it works with any iterable input.

When looking at statistical processing, for example, the enumerate() function comes in handy to transform a single sequence of values into a...

4.8 Summary

In this chapter, we saw detailed ways to use a number of built-in reductions.

We’ve used any() and all() to do essential logic processing. These are tidy examples of reductions using a simple operator, such as or or and. We’ve also looked at numeric reductions such as len() and sum(). We’ve applied these functions to create some higher-order statistical processing. We’ll return to these reductions in Chapter 6, Recursions and Reductions.

We’ve also looked at some of the built-in mappings. The zip() function merges multiple sequences. This leads us to look at using this in the context of structuring and flattening more complex data structures. As we’ll see in examples in later chapters, nested data is helpful in some situations and flat data is helpful in others. The enumerate() function maps an iterable to a sequence of two-tuples. Each two-tuple has the sequence number at index [0] and the original value at index [1].

The reversed...

4.9 Exercises

This chapter’s exercises are based on code available from Packt Publishing on GitHub. See https://github.com/PacktPublishing/Functional-Python-Programming-3rd-Edition.

In some cases, the reader will notice that the code provided on GitHub includes partial solutions to some of the exercises. These serve as hints, allowing the reader to explore alternative solutions.

In many cases, exercises will need unit test cases to confirm they actually solve the problem. These are often identical to the unit test cases already provided in the GitHub repository. The reader should replace the book’s example function name with their own solution to confirm that it works.

4.9.1 Palindromic numbers

See Project Euler problem number 4, https://projecteuler.net/problem=4. The idea here is to locate a number that has a specific property. In this exercise, we want to look at the question of a number being (or not being) a palindrome.

One way to handle this is to decompose a number...

Join our community Discord space

Join our Python Discord workspace to discuss and know more about the book: https://packt.link/dHrHU