Monitoring with CloudWatch

CloudWatch is an AWS service that can be used for monitoring various infrastructure and application resources running on your AWS cloud. CloudWatch can be used to collect a number of metrics from the AWS resources. This allows you to track these metrics and initiate actions based on the thresholds you set. CloudWatch can also collect log files, generate metrics out of them, and help to monitor log files. You can set alarms on specific events and trigger an action whenever an event occurs. For example, if CPU utilization for a specific instance crosses a threshold of 80%, you can initiate an action to spin up a new instance.

The following topics will be covered in this chapter:

• Introducing CloudWatch
• How Amazon CloudWatch works
• Elements of CloudWatch
• Creating a CloudWatch alarm
• Billing alerts
• CloudWatch dashboards
• Monitoring types – normal and...

Introducing CloudWatch

CloudWatch supports the monitoring of many AWS services, such as EC2 instances, DynamoDB, and Relational Database Service (RDS). You can also generate custom metrics and log files using your own applications and associate them with CloudWatch. Amazon services, such as Auto Scaling, use CloudWatch alarms to automatically scale an environment up or down based on the traffic in an environment.

CloudWatch provides a number of graphs and statistics. It gives your system broad insights into how resources are utilized and how to monitor application performance, and tracks the overall operational health of respective applications in an environment. All this infrastructure and application telemetry data can be used to ensure the smooth functioning of your environment.

How Amazon CloudWatch works

CloudWatch acts as a repository of metrics by collating raw data from various AWS services or applications and converting it into metrics, statistics, and graphs, and facilitates certain actions based on specific data points in metrics. The following diagram shows the high-level architecture of CloudWatch:

Figure 7.1: High-level architecture of CloudWatch

As shown in the preceding diagram, various AWS services and your custom metrics data are stored in CloudWatch. CloudWatch generates various statistical and graphical visualizations out of these metrics, which can be consumed directly from the AWS Management Console or by various other means, including, but not limited to, AWS CLI, API, and custom-built applications. CloudWatch enables the user to set alarms that can trigger certain actions based on the metrics threshold or events. It can send email...

Elements of Amazon CloudWatch

To understand and work with AWS-generated and custom metrics, it is important to understand a few basic concepts and terminologies used with Amazon CloudWatch, which are as follows:

• Namespaces
• Metrics
• Dimensions
• Statistics
• Percentile
• Alarms

Namespaces

CloudWatch namespaces are containers in which metrics for different applications are stored. It is a mechanism to isolate the metrics of different applications from each other. Namespaces ensure that an application's metrics, as well as respective statistical data, are not accidentally mixed up with any other application's metrics. All the AWS services that use CloudWatch to register their metrics use a unique namespace. A namespace name...

Creating a CloudWatch alarm

The following steps describe the process of creating a CloudWatch alarm:

1. Open the CloudWatch console by navigating to https://console.aws.amazon.com/ cloudwatch/ on your browser. It brings you to the CloudWatch dashboard.
2. Click on Alarms:

Figure 7.2: CloudWatch dashboard

3. Click on the Create Alarm button:

Figure 7.3: The Create Alarm button

4. Click on Select metric:

Figure 7.4: Selecting a metric

5. This window shows multiple categories of metrics, depending upon the metrics you have in the account. For example, if you have EC2 instances in the account, it shows EC2 metrics; if you have ELB resources in the account, it shows ELB metrics; and, similarly, it shows different categories of metrics for which you have resources in the account. In the following screenshot, we can see metrics categories for EBS and EC2. Depending upon the requirement, you...

Billing alerts

Just as CloudWatch monitors other AWS resources, it can also monitor the monthly billing charges for an AWS account. You can set the threshold for billing. As soon as the billing amount reaches the threshold or shoots above the specified threshold, it notifies the specified administrators. You need to enable billing alerts before you can configure alerts on billing data. You can enable billing alerts from the AWS account settings. Remember, only the root user can enable the billing alerts; AWS IAM users cannot do that.

The process of enabling billing alerts is as follows:

1. Log into the AWS account with the root user credentials.
2. Click on the account name and My Billing Dashboard:

Figure 7.11: Opening the My Billing dashboard

3. From the left-hand pane, click on Preferences and check Receive Billing Alerts. Optionally, you can enable Receive PDF invoice By Email...

CloudWatch dashboards

Amazon CloudWatch provides a customizable dashboard inside a web console. It can display a set of critical metrics together. You can create multiple dashboards, where each dashboard can focus on providing a distinct view of your environment. You can also create a custom dashboard to view and monitor the selected AWS resources from the same or different regions. It provides a way to get a single view of critical resource metrics and alarms for observing the performance and health of the environment. It gives you the freedom to add, remove, move, resize, and rename the graphs, as well as change the refresh interval of selected graphs in a dashboard. The following figure shows a screenshot of the default dashboard view:

Figure 7.13: CloudWatch Dashboard

Monitoring types – basic and detailed

Amazon CloudWatch monitoring can be broadly categorized into two categories, basic monitoring and detailed monitoring, which are described here:

• Basic monitoring: Basic monitoring is free and collects data at 5-minute intervals. By default, when you provision AWS resources, all AWS resources except ELB and RDS start with a basic monitoring mode. ELB and RDS monitor the resources at 1-minute intervals. For other resources, you can switch the monitoring mode to detailed monitoring.
• Detailed monitoring: Detailed monitoring is chargeable and makes data available at 1-minute intervals. Currently, AWS charges $0.015 per hour, per instance. Detailed monitoring does not change the monitoring on ELB and RDS, which, by default, collects data at 1-minute intervals. Similarly, detailed monitoring does not change the EBS volumes, which are monitored...

CloudWatch best practices

Here are the best practices that we can follow in CloudWatch:

• Monitor AWS resources and hosted applications using CloudWatch. This helps to identify performance bottlenecks and to optimize resource costs.
• Enable billing alerts for an AWS account. This helps you to monitor the monthly costs and keep tabs on them.
• For a better understanding of the CloudWatch visualization, toggle the metrics between UTC and local time.
• By default, CloudWatch provides basic monitoring for resources and records metrics at 5-minute intervals. It is recommended that you use detailed monitoring for critical resources, which records metrics at 1-minute intervals.
• Enable custom metrics where required. For example, you can enable memory monitoring on EC2 instances, which is not part of the default EC2 metrics. Create custom metrics for monitoring application behavior and link...

Summary

• CloudWatch is an AWS service that can be used for monitoring various infrastructures and application resources running on your AWS cloud.
• CloudWatch acts as a repository of metrics by collating raw data from various AWS services or applications and converting it into metrics, statistics, and graphs, and facilitates certain actions based on specific data points in metrics.
• CloudWatch namespaces are containers in which metrics for different applications are stored. It is a mechanism to isolate metrics of different applications from each other.
• Metrics are sets of data collected over a period of time with a specific time interval for quantitative assessment, measurement, and comparison of performance data that is generated by a specific application or a service.
• A dimension in a CloudWatch metric is a mechanism to uniquely identify metrics. It is a name/value pair that is...