5. Many to One: Reducing

Overview

In this chapter, you will learn new techniques for dealing with sequential data. You will learn how to use the reduce function, as well as other reducing techniques that provide greater flexibility for transforming or extracting data from a sequence. We will use the simple form of reduce, use reduce with an initializer and an accumulator, and solve problems requiring a variable-length "window" over a sequence. We will also reduce sequences with functions other than reduce.

By the end of this chapter, you will be able to use reduce with complex accumulators.

Introduction

This chapter is about using Clojure's reduce function and about reducing in general. By that, we mean starting with a sequence and boiling it down to a single thing. ("Reducing" is also cooking term, after all.) map and filter were about taking the sequence you have and turning it into the sequence you want: sequence in, sequence out. But that's not always what we want. Even simple operations on a sequence, such as calculating an average, a sum, or a maximum, cannot be directly calculated this way. That's where reduce, as well as a wider family of functions and patterns, comes in: sequence in, something else out. It's "something else" because the result might be a number, a string, a map, or even another sequence.

In the previous chapter, we saw that functions such as map and filter only look at one element at a time: how should we transform this item? Should we discard this item, or keep it? This is a powerful approach because...

Initializing reduce

Tasks such as adding integers or finding maximum values have a common thread: the input values and the accumulated values are of the same type. When two numbers are added, the result is a number; when a maximum or a minimum is chosen between two numbers, the result is still a number. When we use reduce to add numbers together, the running total is a number just like all the other inputs. In the examples so far, the first function call that reduce makes takes the first two items in the sequence. We can break a reduce call into its successive function calls:

(reduce + [1 2 3 5 7])
(+ 1 2)
(+ 3 3)
(+ 6 5)
(+ 11 7)

We actually don't need the anonymous function that we used in the previous examples, because + takes numbers as arguments, and returns a number:

Figure 5.3: The arguments and the return value are all of the same type

In each of our examples so far, three different things are all of the same type:

• The values in the...

Reducing without reduce

Before we go any further, it's important to point out that sometimes there are other, better options than reduce for taking a sequence and turning it into something non-sequential. Often, this is because Clojure provides functions that do the hard work for us. Sometimes, clever use of Clojure's "sequence-to-sequence" functions can get you the data you need.

As a general rule, it is usually preferable to do as much as possible with functions that can handle lazy sequences before turning to reduce. In some cases, this can be for performance reasons, and in nearly all cases, your code will be easier to write, and, more importantly, to read, if you can stay in the realm of sequences. That said, most solutions will require a little of both. Knowing how to combine the two is an important skill.

zipmap

Clojure's zipmap function is a tool for building a map from two sequences. The first sequence becomes the keys for the new map and...

Summarizing Tennis Scores

In the previous chapter, we were able to generate some summary data from the tennis scores, using filter. If we wanted to know how many matches a particular player had won, we could filter out that player's victories and call count. While this approach works well when we are only interested in one player, it becomes cumbersome if we want more complete data. For example, if we needed to know the number of matches played or won by each of the players in the dataset, we would have to filter, for each query, the entire history of all the matches. The map and filter functions are extremely useful in many situations, but reducing a large collection down into a more compact report is not what they are best for.

Let's suppose that for each player, we need to know the number of matches played, won, and lost. We'll walk through two different ways to solve the problem in Clojure, the first using reduce and the second using group-by, one of Clojure&apos...

Introduction to Elo

In the rest of this chapter, and in some of the following chapters, we are going to be working with the Elo Rating System to develop player ratings and predict match outcomes. The algorithm itself is quite simple and it will allow us to demonstrate how Clojure can be used as a data analysis tool. Since we'll be referring to it a lot, it's worth taking a closer look at how it works. The Elo Rating System was developed by Arpad Elo to rate chess players. The United States Chess Federation began using it in 1960.

Elo ratings work by establishing a score for each player. This score is used to calculate the probable outcome of a match. When the real outcome of the match is known, a player's rating is raised or lowered depending on their performance relative to the probable outcome. In other words, if a beginner with a low rating loses to a higher-rated player, the beginner's rating will not suffer by much, since that outcome was expected. If they...

Summary

With this chapter, we've taken another important step forward in our exploration of Clojure's collections and how to use them to solve problems. Techniques involving collections will always be at the heart of your Clojure programming experience: they will inform how you organize your code, as well as how you choose and design your data structures.

In the next chapter, we will take a look at flexible ways to work with the collections in Clojure.