About this book

Icinga has a very flexible configuration that lets you describe your network and server’s infrastructure, and tell Icinga what services you want to monitor and get uptime reports for. You can customize the monitoring behaviour as well as notification methods with plugins. You can also configure handlers that run automatically when a service goes down.

This book gives you a deep insight into setting up automated monitoring for small-scale to large-scale network and server infrastructures. With rising business around cloud computing services such as SaaS, IaaS, and others; service providers have to increase their network infrastructure with a number of servers and services. You will learn to keep tabs on these services to ensure maximum SLA that is promised to the customers.

Icinga comes with ample example configurations that monitor the Icinga server itself. The book analyzes the default sample configuration. You will learn to monitor public services on remote servers, system health of Linux and Windows servers as well as the network devices. You will also look into how to customize the monitoring mechanism with plugins. You will then move towards alerting methods, how they work, and how they can be customized. At the end of the book, you will have a look into the web interface that gives the current status of the entire infrastructure and some reporting tools.

Publication date:
November 2013
Publisher
Packt
Pages
118
ISBN
9781783282296

 

Chapter 1. Installation and Configuration

Icinga is a scalable open source monitoring system that keeps a close watch on our network and server infrastructure, alerts us of problems and resolutions, and gives uptime reports. Icinga is a fork of the popular Nagios project, which aims to build many necessary features on top of it. Icinga is backward compatible with Nagios, so all of the Nagios configuration and plugins can be reused with Icinga as is.

This chapter covers a brief overview of the Icinga architecture and a basic setup to quickly start monitoring a localhost. We will look into the Icinga installation and gain some insight into the initial configuration, so as to get a basic deployment up and running. At the end of this chapter, we will have a web interface showing the status of various services on a localhost and also through an e-mail alert if one of the services goes down.

 

An overview


Icinga consists of various components. The Icinga Core handles scheduling of checks and processing their results. Service states are determined by the Core, by either running checks periodically (active checks) or checking the results that are being reported by a remote system (passive checks). Depending on the checks results, the service state is determined, and depending on the notification configuration, notifications are sent if there is a change in the service state.

The Core is not aware of the checks being executed, nor the notification method that is being used. It simply forks the processes to execute the check plugins, and also performs the processing of exit codes. The notification methods are defined by commands that invoke the underlying operating system provided commands (such as sendmail), or scripts that use notification services to actually deliver SMS, IMs, and so on; this means that Icinga is not aware of the particular notification methods either, it is only aware of the command/script to invoke or sending the notification.

The Core also provides all sorts of information (downtimes, check results, alert history, command execution logs, and so on) via logfiles, which can be used to generate reports on a web interface, for instance. Custom scripts and add-ons can use this logfile to read the current state, and data can show this information in various ways, which is useful for analysis, writing add-ons, and so on.

The data revealed by Icinga can be used to build various web interfaces to present monitoring state, and to do various actions from the web interface (schedule downtime, add comments, disable checks, and so on). There are already a few web interfaces available (Icinga Classic, Thruk, and so on) that can be plugged into Icinga and used.

There are many add-ons available online that extends Icinga's functionality to suit specific needs. Some available add-ons include NagVis (visualizing monitoring status), PNP4Nagios (graphing), and NConf and NagiosQL (managing configuration from web interface) . You can find a number of add-ons by visiting http://www.monitoringexchange.org/.

Icinga's configuration is stored in regular text files along with configuration objects in a simple-to-understand format. Each target server, checks to run on them, and so on are configuration objects that are related to each other as desired.

 

What to expect?


It is good to keep the outcome of the installation process in mind before beginning to have an overall idea of where we are headed, and what things will look like when we are done. We should expect the following at the end this chapter:

  • Icinga installed with default configuration to monitor itself

  • Icinga has started to monitor the localhost with most common checks (load, disk, number of logged-in users, HTTP, ping, SSH, swap, number of processes, and so on)

  • Icinga web UI showing us the status of above checks

  • E-mail notifications when a check fails

 

Requirements


If you are building Icinga from the source, you should have the GCC compiler and some libraries; Icinga will prompt for these during compilation if they are not found. Apart from that, you would need to have a functioning web server for the web interface.

We will need the following before we can start with the installation:

  • GCC compiler

  • Apache web server

  • Nagios plugins (nagios-plugins package for most distributions)

  • SMTP relay server (for example, Postfix) with proper configuration so that mail/mailx commands work

After we have these, we can move on to the download and installation steps.

 

Download


There are various sources and types of Icinga installers. You can get distribution-specific packages, like DEBs for Debian/Ubuntu, RPMs for Red Hat/CentOS, and so on. You can also get the source code and build packages yourself, or directly compile and install it. The source tarball can be downloaded from Icinga's official website from its Downloads section.

 

Installation


The recommended way of installing Icinga is by distribution packages. Debian (Squeeze/Wheezy) and Ubuntu already have upstream packages available on Launchpad, but the latest stable version may be old (Icinga is still relatively under development, so the latest version is preferred since it will have a lot of bug fixes). Red Hat/CentOS have Icinga packages located in the RepoForge YUM repository (http://repoforge.org/). It may still be useful to build an RPM for yourself, for using backported bug fixes without having to wait for the next release. Icinga's default source tarball already has the SPEC file for RPM creation.

Building an RPM for Red Hat / Centos

This is optional and should be done only if you want to build your own RPM. Otherwise, use the RepoForge repository mentioned previously and skip this section.

Icinga's source tarball has the SPEC file for building RPMs. The procedure to build RPMs remains as usual. Create a non-root user account, and log in with that user to build the RPMs. Do not do this as the root user.

Download the Icinga source tarball and extract it. Install RPM Development Tools using the following command:

$ sudo yum install rpmdevtools

You can also install it using the following command:

$ sudo yum install @development-tools fedora-packager

Now, set up the build folder:

$ rpmdev-setuptree
$ ls ~/rpmbuild
BUILD  RPMS  SOURCES  SPECS  SRPMS

Now, copy the SPEC file from the extracted source to SPECS and the source tarball to SOURCES:

$ cp icinga.spec ~/rpmbuild/SPECS
$ cp icinga-*.tar.gz ~/rpmbuild/SOURCES

Build the final RPM packages using the following commands:

$ cd ~/rpmbuild
$ rpmbuild -bb SPECS/icinga.spec

The preceding command might give some dependency errors. Install the listed packages with yum, and run the last rpmbuild command again. If all the dependencies are fixed, the compilation should start, which usually takes some time to complete. The RPM package will be created inside the RPMS/$arch folder, where $arch is the server's architecture (i386/x86_64).

$ ls ~/rpmbuild/RPMS/x86_64
icinga-1.9.1-1.el6.x86_64.rpm
icinga-idoutils-1.9.1-1.el6.x86_64.rpm
icinga-devel-1.9.1-1.el6.x86_64.rpm
icinga-idoutils-libdbi-mysql-1.9.1-1.el6.x86_64.rpm
icinga-doc-1.9.1-1.el6.x86_64.rpm
icinga-idoutils-libdbi-pgsql-1.9.1-1.el6.x86_64.rpm
icinga-gui-1.9.1-1.el6.x86_64.rpm

The icinga package is the Icinga Core; icinga-doc provides the Icinga offline documentation; icinga-gui provides the Icinga web interface; and icinga-idoutils-* provides an optional component to store Icinga information in a database. Now, install the icinga, icinga-doc, and icinga-gui built RPMs:

$ cd ~/rpmbuild/RPMS/x86_64/
$ sudo rpm –ivh {icinga,icinga-doc,icinga-gui}-*.rpm

The installation should succeed. Now install the Apache and Nagios plugins. The nagios-plugins package provides checking of all the basic check plugins that are commonly used for monitoring:

$ sudo yum install httpd nagios-plugins

Also, install SMTP relay server (Postfix is used in the next example) and use the mail command for sending e-mail alerts:

$ sudo yum install postfix mailx

Using DEB for Ubuntu

Debian packages for Ubuntu are available at https://launchpad.net/ubuntu/+source/icinga, which can be used to install Icinga on an Ubuntu Server.

$ sudo apt-get install icinga icinga-doc

This command should install the Icinga package which is ready for use. Install the Apache and Nagios plugins:

$ sudo apt-get install apache2 nagios-plugins

Also use the postfix and mail commands:

$ sudo apt-get install postfix bsd-mailx

This setup is good enough for now. Let's continue with the next section.

Compiling from the source

Let us see how to compile Icinga from the source using the following steps:

  1. Download the Icinga source tarball and extract it:

    $ tar zxvf icinga-1.9.1.tar.gz
    $ cd icinga-1.9.1
    
  2. Run the configure script:

    $ ./configure --prefix=/usr --disable-idoutils
    

    IDOUtils is an optional module that comes with Icinga but we don't want to use it, so we use the configure script to disable the module compilation.

  3. Compile the source code:

    $ make all
    
  4. This will take some time. After it is done, it's time to install!

    $ sudo make fullinstall
    $ sudo make install-config
    

Icinga is now installed and (almost) ready for use.

We need to create a user named icinga because the Icinga server will run as this user:

$ useradd –m icinga
$ passwd icinga

Since the web server will be run as the apache user (in most cases, this is by default, and is not changed unless required by other applications using Apache), we need to add this user to the icinga group.

$ usermod –a icinga apache

The default web server user may be different for different distributions (for example, www, and so on).

 

Make it work


As mentioned in the Requirements section, make sure you have a proper SMTP relay server (such as Postfix) setup so that the usual mail/mailx commands can work. Verify this by sending yourself a test e-mail with the following command on the server (make sure you replace [email protected] with your own e-mail address):

$ echo "This is a test email" | /bin/mail –s "Test email" [email protected]

Proceed further if it works and you receive an e-mail. Troubleshoot your SMTP server if the preceding command gives an error, or you don't receive an e-mail. Open /etc/icinga/objects/contacts.cfg, and replace [email protected] with your own e-mail address in following line:

By default, notifications are disabled, so we need to enable them first. Open /etc/icinga/objects/localhost.cfg, and comment all occurrences of the following line:

notifications_enabled           0

The configuration files can be commented with a hash (#) or a semicolon (;):

# notification_enabled            0

After the installation step, we now have Icinga Core, the web UI, and the Nagios plugins in place. The configuration to monitor your localhost for common services such as ping check, system load, and disk space is already in place. Now, start the icinga service, and (re)start the apache web server to begin the monitoring and see what's happening in the web interface. This can be done using the following commands.

  • For RedHat/CentOS/Fedora:

    $ service icinga start
    $ service httpd start
    
  • For Ubuntu/Debian:

    $ /etc/init.d/icinga start
    $ /etc/init.d/apache2 start
    

Make sure that there were no errors reported for the preceding commands. You can now access the web interface at http://localhost/icinga. The default authentication credentials are username: icingaadmin and password: icingaadmin.

Tip

If you get a connection refused error, make sure that Apache was started properly. Also check if you have proper firewall settings in place to allow connections to the web server. This should not be a problem if you are already using the Apache server for other purposes.

The web server may give constant "Internal Server Errors", due to some distributions shipping with SELinux which is enabled by default. We need to disable it to have the web functioning properly.

$ setenforce 0

You can change the default password for the icingaadmin user to something of your choice with following command:

$ htpasswd /etc/icinga/passwd icingaadmin

This command will prompt for a new password to be set, type in the password and press Enter, this will save your password. Simply reload the web page, and it will ask for the new password.

You should have an interface similar to the following screenshot:

Icinga Web

Go to the Service Detail link on the left sidebar to see all the hosts (localhost only for now) and the services being monitored. It may take a few minutes for Icinga to schedule and complete the checking of all the services; after which, all of them will go green (given that they all pass, of course).

The next critical part is alerting. Note that we have a HTTP check for localhost (visible on the web interface). You can try to stop the web server and see if you get an e-mail notification. Note that it may take five to ten minutes before you get an e-mail alert. Icinga makes sure that the service is really down, by checking multiple times, before it sends out alerts. Also note that when we stop the web server, the web interface will remain inaccessible for that duration.

At this point, we're done with setting up a very basic monitoring server. The last section before we summarize the chapter provides some insight into the basic configuration options of Icinga, which typically applies to how Icinga operates rather than to what it monitors.

 

An overview of configuration options


Configuration is central to customizing Icinga to fit your needs. It becomes possible to highly customize Icinga since the configuration is completely text-based. There is no out of the box configuration that you can install, and automatically start discovering servers on your infrastructure and monitoring them. You need to carefully add all such configurations for Icinga to monitor your network efficiently. This topic gives you a brief overview on the Icinga configuration.

The Icinga configuration is split into two main types: resource files and object files, both of which are specified in the main configuration file along with other Icinga-wide options. This main file contains directives that determine how Icinga operates. The resource file contains Icinga macros that are user defined, such as paths to custom plugins, passwords, and so on. The object files contain Icinga object definitions, each object corresponding to a host, service, and so on. Icinga has object types (host, service, command, and so on) with parameters to build the monitoring configuration.

All of the configuration files are located inside the /etc/icinga folder. The commonly used main configuration file is named as icinga.cfg, the resource file is resources.cfg (it can be used more than once and is specified in the main configuration file). Object files are under the /etc/icinga/objects folder, which contains object definitions that specify monitoring targets. The names of the object files can be anything, with the content being object definitions, which we will look into later. Following is what the configuration structure looks like:

$ tree /etc/icinga
/etc/icinga
├── cgiauth.cfg
├── cgi.cfg
├── conf.d
├── icinga.cfg
├── modules
├── objects
│ ├── commands.cfg
│ ├── contacts.cfg
│ ├── localhost.cfg
│ ├── notifications.cfg
│ ├── printer.cfg
│ ├── switch.cfg
│ ├── templates.cfg
│ ├── timeperiods.cfg
│ └── windows.cfg
├── passwd
└── resource.cfg

We will look at what each configuration file refers to in the subsequent chapters. You can look up the Icinga documentation at http://docs.icinga.org/latest/en/configmain.html for the main configuration file, to see the list of options that are available to customize.

 

Summary


In this chapter, we looked into the basic overview of Icinga's monitoring architecture, installation, and running Icinga's server and web interface. We also looked into the overview of various aspects of configurations that exist in Icinga. Play around with the web interface and the configuration files to get comfortable with Icinga.

In the next chapter, we will go through and understand how the current configuration to monitor the localhost that is in place works.

About the Author

  • Viranch Mehta

    Viranch Mehta is fresh out of college, with experience of one year at Directi Internet Solutions (www.directi.com) as a Developer in Operations. He has done his B.Tech. (ICT) from Dhirubhai Ambani Institute of Information and Communication Technology (www.daiict.ac.in), Gandhinagar (Gujarat). He has been a student Developer for Google Summer of Code (www.google-melange.com) in 2011 and 2012 with the KDE (www.kde.org) project in college, and a mentor in 2013. The author has been a Linux and system administration enthusiast since his college days. He has built a lot of Homebrew automation programs using bash/python scripting as daily tasks in college. He started developing extensively with the KDE project, working primarily with the Plasma subproject, and later with the KDE Games subproject. He has also worked on Qt (qt-project.org) applications and writes small-scale software using Qt. Apart from the work/hobby projects in college, the author has spent a large amount of time as a professional, building and deploying tools for monitoring and alert management. Directi, being an Internet solutions (such as domain name registration, Windows/Linux hosting, contextual advertising, and so on) provider, has a large-scale server and network infrastructure. He has also briefly contributed to Icinga with a few bug fixes.

    Browse publications by this author

Latest Reviews

(1 reviews total)
Dated, we need a book like this for Icinga2 it is now mature and Icinga1 should be deprecated and avoided for new installations
Book Title
Access this book, plus 7,500 other titles for FREE
Access now