Monitoring performance metrics

Now that we have covered a few options for process monitoring, how do you know if your application is actually doing the work you think it is? On many occasions I've seen a stuck syslog-ng process that appeared to be running, but it just wasn't sending any data. I'm not picking on syslog-ng specifically; all software does this when conditions occur that it isn't designed to deal with.

When talking about Flume data flows, you need to monitor the following:

• Data entering sources is within expected rates

• Data isn't overflowing your channels

• Data is exiting sinks at an expected rates

Flume has a pluggable monitoring framework, but as mentioned at the beginning of the chapter, it is still very much a work in progress. That doesn't mean you shouldn't use it since that would be foolish. It means you'll want to prepare extra testing and integration time whenever you upgrade.

While not covered in the Flume documentation, it is common to enable JMX in your Flume JVM (http...