Metrics configuration

The Hadoop daemons expose runtime metrics, which can be collected using plugins. The old Metrics1 system has been replaced by the new Metrics2 system, which supports the following:

• Metrics collection using multiple plugins

• Better integration with JMX

• Better filters for cutting out noise

Before configuring metrics, it is important to understand which metrics and servlets are supported by each Hadoop version. For example, the servlet at /metrics works only with Metrics1 and the new servlet at /jmx works with both Metrics1 and Metrics2.

We need to configure a source, consumer, and poller for the framework:

• Source or producer: A metric source class must implement the following interface:

  org.apache.hadoop.metrics2.MetricsSource
  

• Consumer or sink: A consumer or sink must be implemented with the following line of code:

  org.apache.hadoop.metrics2.MetricsSink
  

  For example, the configuration for JobTracker sink and filter is as follows:

  jobtracker.sink.file.class=org.apache.hadoop.metrics2...