Defining alert rules

Let’s start off by talking about how we want to look at triggering alerts. To build an alert, you will need to answer a series of questions in this form:

What condition must exist as measured by what metrics, and for how long?

Let’s break this concept down into its constituent parts.

What condition…

An alert ultimately boils down to a switch: at any given moment in time, the evaluation interval, an alert may need to be triggered. How you determine whether the alert should be in a triggered (or firing) state is called the alert condition. Most of the work you will do in defining an alert condition consists of reducing metrics data to a simple Boolean yes-or-no assertion about whether an alert should be triggered.

Space prevents us from devoting an entire chapter to exploring the possible ways to define alert conditions, but I can offer some heuristics for identifying possible alert conditions:

• Is the condition based upon...