Identifying slow computation

There are many reasons why a particular piece of code might be slower than expected. Most of the time, this is because of memory bottlenecks and suboptimal cache performance. However, one can also run into other problems, such as instruction cache overflows and branch prediction failures. Instruction cache overflows occur when your program contains too many instructions for the processor to keep in cache at any given time. For instance, this might occur if one unrolls a loop into too many steps. Each time the outer loop returns to the beginning, this will cause an instruction cache miss and greatly reduce the ability of the instruction decode to feed instructions to the core itself. perf sometimes refers to issues involving instruction decoding and branching as being frontend-bound.

In Chapter 4, we discussed in detail how speculative execution is one of the reasons modern processors can perform so well, and gave an example of where speculative execution...