Reader small image

You're reading from  Monitoring Hadoop

Product typeBook
Published inApr 2015
Publisher
ISBN-139781783281558
Edition1st Edition
Tools
Right arrow
Author (1)
Aman Singh
Aman Singh
author image
Aman Singh

Gurmukh Singh is a seasoned technology professional with 14+ years of industry experience in infrastructure design, distributed systems, performance optimization, and networks. He has worked in big data domain for the last 5 years and provides consultancy and training on various technologies. He has worked with companies such as HP, JP Morgan, and Yahoo. He has authored Monitoring Hadoop by Packt Publishing
Read more about Aman Singh

Right arrow

Hadoop Ganglia integration


Ganglia is a metrics collection and a visualization tool for the enterprise and works very well with Nagios and Hadoop. In addition to just collecting stats about CPU, memory, and disk, other finely tuned metrics are required, which can be provided by this framework.

Until now, we have seen that the metrics collection can be done to a file or to any other tool like Splunk, depending upon the class interface. We can configure which class handles the metrics update.

For Ganglia, we use GangliaContext, which is an implementation of MetricsContext. Ganglia versions higher than 3.0 provide this integration and work very well for collecting the Hadoop metrics.

In Ganglia, the metrics can be collected for NameNode, JobTracker, MapReduce tasks, JVM, RPC, DataNodes, and the new YARN framework.

Hadoop metrics configuration for Ganglia

Firstly, we need to define a sink class, as per Ganglia version 3.1:

*.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31

Secondly...

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Monitoring Hadoop
Published in: Apr 2015Publisher: ISBN-13: 9781783281558

Author (1)

author image
Aman Singh

Gurmukh Singh is a seasoned technology professional with 14+ years of industry experience in infrastructure design, distributed systems, performance optimization, and networks. He has worked in big data domain for the last 5 years and provides consultancy and training on various technologies. He has worked with companies such as HP, JP Morgan, and Yahoo. He has authored Monitoring Hadoop by Packt Publishing
Read more about Aman Singh