Zabbix 1.8 Network Monitoring

Chapter 1. Getting Started with Zabbix

It's Friday night, and you are at a party outside the city with old friends. After a few beers it looks like this is going to be a great party, when suddenly your phone rings. A customer can't access some critical server that absolutely has to be available as soon as possible. You try to ssh in the server, only to discover that customer is right it can't be accessed.

As driving after those few beers would quite likely lead to inoperable server for quite some time, you get a taxi (expensive because of the distance. While many modern systems have out-of-bands management cards installed that might have helped a bit in such a situation, our hypothetical administrator does not have one available). After arriving at the server room, you find out that some logfiles have been growing more than usual over the past few weeks and have filled up the hard drive.

While the scenario above is very simplistic, something similar has probably happened to most IT workers at one or another point in their careers. Most implemented a simple system monitoring and reporting solution soon after that.

We will learn to set up and configure one such monitoring system - Zabbix.

First steps in monitoring

Situations similar to the one described above, are actually more common than desired. A system fault that had no symptoms visible before is relatively rare. Probably a subsection of Unix Administration Horror Stories (visit http://www-uxsup.csx.cam.ac.uk/misc/horror.txt) could be easily compiled that contained only stories about faults that were not noticed on time.

As experience shows, problems tend to happen when we are least equipped to solve them. To work with them on our terms we turn to a class of software, commonly referred to as network monitoring software. Such software usually allows us to constantly monitor things happening in a computer network using one or more methods and notify the persons responsible if some metric passes a defined threshold.

One of the first monitoring solutions most administrators implement is a simple shell script, invoked from crontab, that checks some basic parameters like disk usage or some service state, like Apache server. As the server and monitored parameter count grows, a neat and clean script systems starts to grow into a performance-hogging script hairball that costs more time in upkeep than it saves. While do-it-yourself crowds claim that nobody needs dedicated software for most tasks (monitoring included), most administrators will disagree as soon as they have to add switches, UPSes, routers, IP cameras, and a myriad of other devices to the swarm of monitored objects.

So what basic functionality can one expect from a monitoring solution? They are as follows:

Data gathering: This is where everything starts. Usually data will be gathered using various methods, including SNMP, agents, IPMI, and others.
Alerting: Gathered data can be compared to thresholds and alerts sent out when needed using different channels, like e-mail or SMS.
Data storage: Once we have gathered the data it doesn't make sense to throw it away, so we will often want to store it for later analysis.
Visualization: Humans are better at distinguishing visualized data than raw numbers, especially when there are huge amounts of them. As we have data already gathered and stored, it is trivial to generate simple graphs from it.

Sounds simple? That's because it is. But then we start to desire more features like easy and efficient configuration, escalations, permission delegation, and so on. If we sit down and start listing the things we want to keep an eye out for, it may turn out that area of interest extends beyond the network for example, a hard drive that has SMART errors logged, an application that has too many threads, or a UPS that has one phase overloaded. It is much easier to manage monitoring of all these different problem categories from a single configuration point.

In the quest for a manageable monitoring system wondrous adventurers stumbled upon collections of scripts much like the way they implemented themselves, obscure and not so obscure workstation-level software, and heavy, expensive monitoring systems from big vendors.

Another group is open source monitoring systems that have various sophistication levels, one of which is Zabbix.

First steps in monitoring

So what basic functionality can one expect from a monitoring solution? They are as follows:

Data gathering: This is where everything starts. Usually data will be gathered using various methods, including SNMP, agents, IPMI, and others.
Alerting: Gathered data can be compared to thresholds and alerts sent out when needed using different channels, like e-mail or SMS.
Data storage: Once we have gathered the data it doesn't make sense to throw it away, so we will often want to store it for later analysis.
Visualization: Humans are better at distinguishing visualized data than raw numbers, especially when there are huge amounts of them. As we have data already gathered and stored, it is trivial to generate simple graphs from it.

Another group is open source monitoring systems that have various sophistication levels, one of which is Zabbix.

Zabbix features and architecture

Zabbix provides many ways to monitor different aspects of your IT infrastructure and indeed, almost anything one might want to hook to it. It can be characterized as a semi-distributed monitoring system with centralized management. While many installations have a single central database, it is possible to use distributed monitoring with nodes and proxies, and most installations will use Zabbix agents.

So what features does Zabbix provide? They are:

Centralized, easy to use web interface
Server that runs on most Unix-like operating systems, including Linux, AIX, FreeBSD, OpenBSD, and Solaris
Native agents for most Unix-like operating systems and Microsoft Windows versions
Ability to directly monitor SNMP (v1, 2, and 3) and IPMI devices
Built-in graphing and other visualization capabilities
Notifications that allow for easy integration with other systems
Flexible configuration, including templating
And a lot of other features that would allow you to implement a sophisticated monitoring solution

If we look at a simplified network from the Zabbix perspective, placing Zabbix server at the center, the communication of the various monitoring aspects matters. The following image depicts a relatively simple Zabbix setup with several of the monitoring capabilities used and different device categories connected.

Our central object is the Zabbix database, with several backends supported. Zabbix server, written in C, and web frontend written in PHP, can both reside on the same machine or on another server. When running each component on a separate machine, both the Zabbix server and the frontend need access to the database, and frontend optionally needs access to Zabbix server to show server status. Required connection directions are depicted by arrows in the following image.

Zabbix server directly monitors multiple devices, but a remote location is separated by a firewall, so it gathers data through a Zabbix proxy. Zabbix proxy and agents, just like the server, are written in C.

While it is perfectly fine to run all three server components on a single machine, there might be good reasons to separate them, like taking advantage of an existing high performance database or web server.

In general, monitored devices have little control over what is monitored most of the configuration is centralized. Such an approach seriously reduces the capabilities of single misconfigured system to bring down the whole monitoring setup.

Installation

Alright, enough with the dry-talk, what will we get? Let's look at dashboard screen of Zabbix web frontend, showing only a very basic configuration.

As we can see, Zabbix dashboard shows a high level overview of overall monitored system status, status of Zabbix, some of the most recent problems, and a few more things. This particular dashboard shows a very tiny Zabbix setup. Eventually your Zabbix installation will grow and provide monitoring of different devices, including servers of various operating systems, different services and hardware state on those servers, network devices, UPSes, web pages, other components of IT, and other infrastructure.

The frontend will provide various options for visualizing data, starting from problem lists and simple graphs ending with network maps and reports, while backend will work hard to provide information that this visualization is based on and send out alerts. All of this will require some configuration that we will learn to perform along the course of this book.

Before we can configure Zabbix, we need to install it. Usually you'll have two choices - either installing from distribution packages, or setting it up from the source code. Unless you highly value distribution packaging and are not ready to roll your own packages, it is suggested to set up the latest version from sources, because Zabbix is being developed at a relatively noticeable pace and there's always some neat feature in the next version that makes life easier.

At first we will set up Zabbix server, database, and frontend, all running on the same machine and using a MySQL database.

If you decide to install Zabbix from your distribution packages, installation procedure and package naming schemes will differ. Refer to your distribution's documentation for that information.

There are a few benefits to using distribution packages. These include:

Automated installation and updating
Dependencies usually sorted out

Compiling from source also has its share of benefits. They are:

Newer versions with more features and improvements
More fine-grained control over compiled-in functionality

Server and agent

The most widely-used Zabbix architecture is a server that queries agents. That's what we will learn to set up so that we can monitor our test system.

As with most software, there are some prerequisites that we will need to run Zabbix components. That includes requirements for hardware and other software that the Zabbix server and agent depend on. For the purpose of these instructions, we will settle on running Zabbix on Linux, using a MySQL database. The specific Linux distribution does not matter much it's best to choose the one you are most familiar with.

Software requirements

Now we should get to compiling the various components of Zabbix, so make sure to install the minimum required packages to get Zabbix working with MySQL. They are:

GCC
Automake
MySQL (http://www.mysql.com/)

Depending on your distribution and the desired functionality you might also need some or all of the following packages:

zlib-devel
mysql-devel (for MySQL support)
glibc-devel
curl-devel (for web monitoring)
libidn-devel (curl-devel might depend on it)
openssl-devel (curl-devel might depend on it)
net-snmp-devel (for SNMP support)
popt-devel (net-snmp-devel might depend on it)
rpm-devel (net-snmp-devel might depend on it)
OpenIPMI-devel (for IPMI support)
libssh2-devel (for direct SSH checks)

Hardware requirements

Hardware requirements vary wildly depending on the configuration. It is impossible to give definite requirements, so any production installation should evaluate them individually. For our test environment, though, even as little RAM as 128 MB should be enough. CPU power in general won't play a huge role; Pentium II class hardware should be perfectly capable of dealing with it, although generating graphs with many elements or other complex views can require more powerful hardware to operate at an acceptable speed. You can take these as a starting point as well when installing in a virtual machine.

Of course, the more resources you give to Zabbix, the snappier and happier it will be.

Getting the source

There are several ways to download the source of Zabbix. You can get the source code from a SVN repository, which will be discussed in Appendix B, however for this installation procedure it is suggested to download version 1.8.1 from the Zabbix homepage; http://www.zabbix.com/. While it should be possible to use latest stable version, using 1.8.1 will allow to follow instructions more closely. Go to the Download section and grab the compressed source package. Usually only the latest stable version is available on the downloads page, so you might have to browse the source archives, though do not take development or beta version, which might be available.

To ease further references, it is suggested that you choose a directory to work in, for example, ~/zabbix (~ being your home directory). Download the archive into this directory.

Compilation

Once the archive has finished downloading, open a terminal and extract it:

$ cd ~/zabbix; tar -zxvf zabbix-1.8.1.tar.gz

It is suggested that you install the requirements and compile Zabbix with external functionality right away so that you don't have to recompile as we progress.

For the purpose of this book, we will compile Zabbix with server, agent, MySQL, curl, SNMP, and IPMI support.

To continue, enter the following in the terminal:

$ cd zabbix-1.8.1
$ ./configure --enable-server --with-mysql --with-net-snmp --with-libcurl --with-openipmi --enable-agent

In the end, a summary of compiled components is printed. Verify that you have the following enabled:

Enable server: yes
With database: MySQL
WEB Monitoring via: cURL
SNMP: net-snmp
IPMI: openipmi
Enable agent: yes

If configure completes successfully, it's all good. If it fails, check the error messages printed in the console and verify that all prerequisites are installed. A file named config.log might provide more detail on the errors. If you can't find out what's wrong, check Appendix A, Troubleshooting, which lists some common compilation problems.

To actually compile Zabbix, issue the next command.

$ make

You can get a cup of tea, but don't expect to have much time - Zabbix compilation doesn't take too long, even an old 350 MHz Pentium II compiles it in approximately five minutes. After the make process has finished, check the last lines for any error messages. If there are none, congratulations, you have successfully compiled Zabbix.

Now we should install it. Despite the output of configure and many How Tos suggesting so, do not run make install. It unnecessarily complicates package management and can lead to weird problems in future. Instead, you should create proper packages for your distribution. This process is package system specific, learning them all would be cumbersome, but there's a universal solution - software named CheckInstall (http://www.asic-linux.com.mx/~izto/checkinstall/). Packages should be available for most distributions, and it supports the creation of Slackware, RPM, and Debian packages.

To create a proper Zabbix package make sure that CheckInstall is installed and execute as root:

# checkinstall --nodoc --install=yes -y

This will create and install a package that you will later be able to remove using your system's package management tool. The created package's location depends on your distribution for Slackware, it's the directory you executed CheckInstall from, for RPM-based distributions it is usually /usr/src/packages/RPMS/<architecture>.

Initial configuration

After compilation, we have to configure some basic parameters for the server and agent. There are example configuration files provided with the Zabbix package, so let's use those. Again, as root execute:

# mkdir /etc/zabbix
Zabbix sourceinitial configuration# cp misc/conf/{zabbix_server.conf,zabbix_agentd.conf} /etc/zabbix

To configure the Zabbix agent, we don't have to do anything. The default configuration will do just fine for now. That was easy, right?

For the server we will need to make some changes. Open /etc/zabbix/zabbix_server.conf in your favorite editor (you will need to run it as root) and find the following entries in the file:

DBName
DBUser
DBPassword

DBName should be zabbix by default, and we can leave it as is. DBUser is set to root, and we don't like that, so let's change it to zabbix. For DBPassword, choose any password. You won't have to remember it, so be creative.

After we insert a password into this configuration file, we should restrict the file's permissions so that the password is not that easy to obtain. This is done as shown below:

# chmod 400 /etc/zabbix/zabbix_server.conf
# chown zabbix /etc/zabbix/zabbix_server.conf

Creating and populating the database

For the Zabbix server to store data, we have to create a database. Start a MySQL client.

$ mysql -u root -p

Enter the root user password for MySQL (you would have set this during the installation of MySQL, or the password could be something that is the default for your distribution). If you do not know the password, you can try omitting -p. This switch will tell the client to attempt to connect with an empty password.

Now let's create the database. Add the user that Zabbix would connect to the database as, and grant necessary permissions to this user:

mysql> create database zabbix character set utf 8;

Query OK, 1 row affected (0.01 sec)
mysql> grant all privileges on zabbix.* to 'zabbix'@'localhost' identified by 'mycreativepassword';

Query OK, 0 rows affected (0.12 sec)

Use the same password you set in zabbix_server.conf file instead of mycreativepassword.

Quit the MySQL client by entering the following command:

mysql> quit

Let's populate the newly-created database with Zabbix schema and initial data.

$ mysql -u zabbix -p zabbix < create/schema/mysql.sql
Zabbix sourcedatabase, populating$ mysql -u zabbix -p zabbix < create/data/data.sql

Next, let's insert the images to be used in network maps. While these images are not required for basic functionality, we'll want to create some nice looking network maps later.

$ mysql -u zabbix -p zabbix < create/data/images_mysql.sql

All three importing processes should complete without any messages. If there are any errors, review the messages, fix the issue, and retry the failed operation. Note, if the import is interrupted in the middle of the process, you might have to clear the database easiest way to do that is deleting the database by typing:

mysql> drop database zabbix;

Query OK, 0 rows affected (0.00 sec)

Be careful not to delete any database with important information! After deleting the Zabbix database recreate it and assign the correct user permissions, as indicated above.

By now we should have Zabbix server and agent installed, and ready to start.

Starting up

You should never start the Zabbix server or agent as root, which is common sense for most daemon processes, so let's create user to run these processes. You can use tools provided by distribution, or the most widely available command - useradd, which we need to execute as root:

# useradd -m -s /bin/bash zabbix

This will create user named zabbix with a home directory in the default location (usually /home/zabbix) and shell at /bin/bash, which will be needed for advanced parameters later on.

For the first startup of both server and agent let's try the direct approach as root execute:

# /usr/local/sbin/zabbix_agentd

This will start the Zabbix agent daemon, which should start up silently and daemonize. If the above command produces errors, resolve those before proceeding. If it succeeds, continue by starting the Zabbix server.

# /usr/local/sbin/zabbix_server

Note

We are using zabbix_agentd that runs as a daemon. While there's also the zabbix_agent executable that provides an option to be run within inetd, it does not support active items and in most cases will have worse performance than the agent daemon.

If you decided to install distribution packages of Zabbix above binaries will most likely be located in a different directory, but they should be in your path so try running the agent and server without specifying a directory name.

Note

The latest versions of Zabbix, including 1.8, automatically drop root privileges on startup and run as the zabbix user.

While it's nice to have Zabbix running, that's hardly a process one expects to do manually upon each system boot, so the server and agent should be added to your system's startup sequence. This is fairly distribution specific, so all possible variations can't be discussed here. Instead, examples for SUSE Linux Enterprise Server and Slackware startup scripts will be provided.

SUSE Linux Enterprise Server

The Zabbix package provides example startup scripts, but they are somewhat outdated. Here are init scripts for Zabbix server and agent daemons, based on the SUSE Linux Enterprise Server 10 SP2 skeleton file. It should be possible to use these files on most other distributions that have SysV-style init scripts. For Zabbix server, place the following in the /etc/init.d/zabbix_server file:

#! /bin/sh
#
# Original skeleton file from SLES10 Sp2 - # Copyright (C) 1995--2005 Kurt Garloff, SUSE / Novell Inc.
# Zabbix specifics - # Copyright (C) 2008--2009 Richlv
# This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public # License as published by the Free Software Foundation; # either version 2.1 of the License, or (at your option) # any later version.
# This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # See the GNU Lesser General Public License for more details.
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the # Free Software Foundation, Inc., 59 Temple Place, Suite 330, # Boston, MA 02111-1307, USA.
# /etc/init.d/zabbix_server
# and its symbolic link
# /usr/sbin/rczabbix_server
#
### BEGIN INIT INFO
# Provides: zabbix_server
# Required-Start: $network $remote_fs $local_fs
# Should-Start: mysql postgresql
# Required-Stop: $network $remote_fs $local_fs
# Should-Stop: mysql postgresql
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Short-Description: Zabbix monitoring server daemon
# Description: This is a server daemon for the monitoring system Zabbix
# For more information see http://www.zabbix.com
### END INIT INFO
ZABBIX_CONFIG="/etc/zabbix/zabbix_server.conf"
test -r $ZABBIX_CONFIG || { echo "$ZABBIX_CONFIG missing";
if [ "$1" = "stop" ]; then exit 0;
else exit 6; fi; }
SUSE Linux Enterprise Serverstratup scriptsZABBIX_BIN="/usr/local/sbin/zabbix_server"
test -x $ZABBIX_BIN || { echo "$ZABBIX_BIN not installed";
if [ "$1" = "stop" ]; then exit 0;
else exit 5; fi; }
. /etc/rc.status
# Reset status of this service
rc_reset
NAME="Zabbix server daemon"
ZABBIX_PID=/var/tmp/zabbix_server.pid
case "$1" in
start)
echo -n "Starting $NAME "
## Start daemon with startproc(8). If this fails
## the return value is set appropriately by startproc.
/sbin/startproc -t 1 -p $ZABBIX_PID $ZABBIX_BIN
# Remember status and be verbose
rc_status -v
;;
stop)
echo -n "Shutting down $NAME "
## Stop daemon with killproc(8) and if this fails
## killproc sets the return value according to LSB.
/sbin/killproc -TERM $ZABBIX_BIN
# Remember status and be verbose
rc_status -v
;;
try-restart|condrestart)
## Do a restart only if the service was active before.
## Note: try-restart is now part of LSB (as of 1.9).
## RH has a similar command named condrestart.
if test "$1" = "condrestart"; then
echo "${attn} Use try-restart ${done}(LSB)${attn} rather than condrestart ${warn}(RH)${norm}"
fi
$0 status
if test $? = 0; then
$0 restart
SUSE Linux Enterprise Serverstratup scriptselse
rc_reset # Not running is not a failure.
fi
# Remember status and be quiet
rc_status
;;
restart)
## Stop the service and regardless of whether it was
## running or not, start it again.
$0 stop
$0 start
# Remember status and be quiet
rc_status
;;
force-reload)
## Zabbix server daemon does not support configuration reloading, thus it is restarted, if running.
echo -n "Reload service $NAME "
$0 try-restart
rc_status
;;
reload)
## Zabbix server daemon does not support configuration reloading, thus reload fails
rc_failed 3
rc_status -v
;;
status)
echo -n "Checking for service $NAME "
## Check status with checkproc(8), if process is running
## checkproc will return with exit status 0.
# Return value is slightly different for the status command:
# 0 - service up and running
# 1 - service dead, but pid file exists
# 2 - service dead, but /var/lock/ lock file exists
# 3 - service not running (unused)
# 4 - service status unknown :-(
# 5--199 reserved (5--99 LSB, 100--149 distro, 150--199 appl.)
# NOTE: checkproc returns LSB compliant status values.
/sbin/checkproc -p $ZABBIX_PID $ZABBIX_BIN
# NOTE: rc_status knows that we called this init script with
# "status" option and adapts its messages accordingly.
rc_status -v
SUSE Linux Enterprise Serverstratup scripts;;
probe)
## Optional: Probe for the necessity of a reload, print out the
## argument to this init script which is required for a reload.
## Note: probe is not (yet) part of LSB (as of 1.9)
test $ZABBIX_CONFIG -nt $ZABBIX_PID && echo reload
;;
*)
echo "Usage: $0 {start|stop|status|try-restart|restart|force- reload|reload|probe}"
exit 1
;;
esac
rc_exit

For the Zabbix agent daemon, create /etc/init.d/zabbix_agentd with the following contents:

#! /bin/sh
#
# Original skeleton file from SLES10 Sp2 - # Copyright (C) 1995--2005 Kurt Garloff, SUSE / Novell Inc.
# Zabbix specifics - # Copyright (C) 2008--2009 Richlv
#
# This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public # License as published by the Free Software Foundation; either # version 2.1 of the License, or (at your option) any # later version.
#
# This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # See the GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the # Free Software Foundation, Inc., 59 Temple Place, Suite 330, # Boston, MA 02111-1307, USA.
SUSE Linux Enterprise Serverstratup scripts#
# /etc/init.d/zabbix_agentd
# and its symbolic link
# /usr/sbin/rczabbix_agentd
#
### BEGIN INIT INFO
# Provides: zabbix_agentd
# Required-Start: $network $remote_fs $local_fs
# Required-Stop: $network $remote_fs $local_fs
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Short-Description: Zabbix monitoring agent daemon
# Description: This is a client daemon for the monitoring system Zabbix
# For more information see http://www.zabbix.com
### END INIT INFO
ZABBIX_CONFIG="/etc/zabbix/zabbix_agentd.conf"
test -r $ZABBIX_CONFIG || { echo "$ZABBIX_CONFIG missing";
if [ "$1" = "stop" ]; then exit 0;
else exit 6; fi; }
ZABBIX_BIN="/usr/local/sbin/zabbix_agentd"
test -x $ZABBIX_BIN || { echo "$ZABBIX_BIN not installed";
if [ "$1" = "stop" ]; then exit 0;
else exit 5; fi; }
. /etc/rc.status
# Reset status of this service
rc_reset
NAME="Zabbix agent daemon"
ZABBIX_PID=/var/tmp/zabbix_agentd.pid
case "$1" in
start)
echo -n "Starting $NAME "
## Start daemon with startproc(8). If this fails
## the return value is set appropriately by startproc.
/sbin/startproc -t 1 -p $ZABBIX_PID $ZABBIX_BIN
# Remember status and be verbose
rc_status -v
;;
stop)
echo -n "Shutting down $NAME "
## Stop daemon with killproc(8) and if this fails
## killproc sets the return value according to LSB.
/sbin/killproc -TERM $ZABBIX_BIN
# Remember status and be verbose
rc_status -v
SUSE Linux Enterprise Serverstratup scripts;;
try-restart|condrestart)
## Do a restart only if the service was active before.
## Note: try-restart is now part of LSB (as of 1.9).
## RH has a similar command named condrestart.
if test "$1" = "condrestart"; then
echo "${attn} Use try-restart ${done}(LSB)${attn} rather than condrestart ${warn}(RH)${norm}"
fi
$0 status
if test $? = 0; then
$0 restart
else
rc_reset # Not running is not a failure.
fi
# Remember status and be quiet
rc_status
;;
restart)
## Stop the service and regardless of whether it was
## running or not, start it again.
$0 stop
$0 start
# Remember status and be quiet
rc_status
;;
force-reload)
## Zabbix agent daemon does not support configuration reloading, thus it is restarted,
## if running.
echo -n "Reload service $NAME "
$0 try-restart
rc_status
;;
reload)
## Zabbix agent daemon does not support configuration reloading, thus reload fails
rc_failed 3
rc_status -v
;;
status)
echo -n "Checking for service $NAME "
## Check status with checkproc(8), if process is running
## checkproc will return with exit status 0.
SUSE Linux Enterprise Serverstratup scripts# Return value is slightly different for the status command:
# 0 - service up and running
# 1 - service dead, but pid file exists
# 2 - service dead, but /var/lock/ lock file exists
# 3 - service not running (unused)
# 4 - service status unknown :-(
# 5--199 reserved (5--99 LSB, 100--149 distro, 150--199 appl.)
# NOTE: checkproc returns LSB compliant status values.
/sbin/checkproc -p $ZABBIX_PID $ZABBIX_BIN
# NOTE: rc_status knows that we called this init script with
# "status" option and adapts its messages accordingly.
rc_status -v
;;
probe)
## Optional: Probe for the necessity of a reload, print out the
## argument to this init script which is required for a reload.
## Note: probe is not (yet) part of LSB (as of 1.9)
test $ZABBIX_CONFIG -nt $ZABBIX_PID && echo reload
;;
*)
echo "Usage: $0 {start|stop|status|try-restart|restart|force- reload|reload|probe}"
exit 1
;;
esac
rc_exit

To use these scripts, place them in /etc/init.d and create appropriate symlinks in /usr/sbin as root:

# ln -s /etc/init.d/zabbix_server /usr/sbin/rczabbix_server
# ln -s /etc/init.d/zabbix_agentd /usr/sbin/rczabbix_agentd

Now let's try to start our newly created services. Again, as root execute:

# service zabbix_agentd start

Starting zabbix agent daemon done
# service zabbix_server start

Starting zabbix server daemon done

That should get the agent and server started, although there should be no additional output. Note: you can also use other syntax remember those symlinks we just created? You can also use them as:

# rczabbix_agentd start
# rczabbix_server start

Feel free to experiment with other parameters, like stop and restart it should be obvious what those two do.

You can verify whether services are running with the status parameter. For a service that is not running, you would get:

# service zabbix_server status

Checking for service Zabbix server daemon unused

A running service would yield:

# service zabbix_agentd status

Checking for service Zabbix agent daemon running

There's also another parameter called probe. This will check whether the corresponding service has been restarted since the last configuration file changes. If it has been restarted, no output will be produced. If the service has not been restarted (thus possibly missing some configuration changes), the string reload will be output.

Now that's nice, but we started all this with the goal of adding Zabbix services to system startup. Surely the previous actions didn't help with that? To reach our goal when using SysV-style init systems input the following as root:

# chkconfig -s zabbix_server 35
# chkconfig -s zabbix_agentd 35

This will add both services to be started at runlevels 3 and 5, which are used for multiuser and networked environments (though some distributions might use runlevel 4 instead of 5 for a graphical environment; consult your distribution's documentation when in doubt). There's usually no need to start Zabbix in single user or non-networked runlevels (1 and 2), as data gathering requires network connectivity.

To be fair, with the init scripts we created earlier it is even simpler than that:

# chkconfig -a zabbix_server zabbix_agentd

This will add both services as specified in the corresponding init scripts, which in our case is runlevels 3 and 5, configured by parameter Default-Start. If the command succeeds, you'll see the following output:

zabbix_server 0:off 1:off 2:off 3:on 4:off 5:on 6:off
zabbix_agentd 0:off 1:off 2:off 3:on 4:off 5:on 6:off

Slackware

The Slackware Linux distribution (http://www.slackware.com/) uses a simpler startup script system. Here's an example of a script to start and stop the Zabbix server and agent daemon on Slackware systems. This script is dependant on bash, but it shouldn't be too hard to rewrite for sh compatibility, if so desired. Create script /etc/rc.d/rc.zabbix with the following content:

#!/bin/bash
# Init script to start/stop Zabbix server and agent daemon
BINLOCATION=/usr/local/sbin
AGENTPID=/var/tmp/zabbix_agentd.pid
SERVERPID=/var/tmp/zabbix_server.pid
TERMINATEWAIT=5
processcheck() {
[[ "$(ps -C $1 -o pid=)" ]] || return 1
}
# Check for stray pidfiles and remove them
processcheck zabbix_agentd || {
[ -f $AGENTPID ] && ( echo "Removing stray $AGENTPID file"; rm $AGENTPID )
}
processcheck zabbix_server || {
[ -f $SERVERPID ] && ( echo "Removing stray $SERVERPID file"; rm $SERVERPID )
}
killprocess() {
processcheck $2 || {
echo "No process $2 running"
Slackwarescript, creatingreturn
}
if [ -f $1 ]; then
kill -15 $(cat $1)
else
echo "pidfile $1 not found"
processcheck $2 && killall -15 $2
fi
echo -n "Waiting for $2 to terminate"
for i in $(seq 1 $TERMINATEWAIT); do
processcheck $2 || break
sleep 1
echo -n "."
done
processcheck $2 && echo "Warning! $2 did not terminate in $TERMINATEWAIT seconds"
}
zagent_start() {
if [ -x $BINLOCATION/zabbix_agentd ]; then
if processcheck zabbix_agentd; then
echo "Zabbix agent daemon already running"
else
echo "Starting zabbix agent daemon: $BINLOCATION/zabbix_agentd"
$BINLOCATION/zabbix_agentd
fi
else
echo "Executable $BINLOCATION/zabbix_agentd not present"
fi
}
zserver_start() {
if [ -x $BINLOCATION/zabbix_server ]; then
if processcheck zabbix_server; then
echo "Zabbix server already running"
else
echo "Starting zabbix server: $BINLOCATION/zabbix_server"
$BINLOCATION/zabbix_server
fi
else
echo "Executable $BINLOCATION/zabbix_server not present"
fi
}
zagent_stop() {
Slackwarescript, creatingecho "Stopping Zabbix agent daemon"
killprocess $AGENTPID zabbix_agentd
}
zserver_stop() {
echo "Stopping zabbix server"
killprocess $SERVERPID zabbix_server
}
zagent_restart() {
zagent_stop
zagent_start
}
zserver_restart() {
zserver_stop
zserver_start
}
case "$1" in
'start')
case "$2" in
'agent')
zagent_start
;;
'server')
zserver_start
;;
*)
zagent_start
zserver_start
;;
esac
;;
'stop')
case "$2" in
'agent')
zagent_stop
;;
'server')
zserver_stop
;;
*)
zagent_stop
zserver_stop
;;
esac
;;
'restart')
case "$2" in
'agent')
zagent_restart
Slackwarescript, creating;;
'server')
zserver_restart
;;
*)
zagent_restart
zserver_restart
;;
esac
;;
*)
echo "Usage: $0 start|stop|restart [agent|server]"
;;
esac

This script combines agent and server controlling. To start both, execute the following as root:

# /etc/rc.d/rc.zabbix start

To control the server or agent only, pass its name as a second argument. For example:

# /etc/rc.d/rc.zabbix restart server

Adding a script to the system's startup sequence is quite trivial. It's invocation has to be added to the /etc/rc.d/rc.local as follows:

# echo /etc/rc.d/rc.zabbix start >> /etc/rc.d/rc.local

Verifying the service's state

While the init script method is a nice way to check a service's state for some distributions, it's not available everywhere and isn't always enough. Sometimes you might want to check whether the Zabbix server or agent is running with other methods.

Checking running processes:
The most common method to check whether a particular process is running is by looking at running processes. You can verify if Zabbix agent daemon processes are actually running by using the following:
```
$ ps -C zabbix_agentd
```
netstat output:
Sometimes, an agent daemon might start up, but fail to bind to the port, or the port might be used by some other process. You can verify whether some other process is listening on the Zabbix port, or whether Zabbix agent daemon is listening on the correct port by issuing:
```
$ netstat -ntpl
```
Note that process names won't be printed for other users' processes unless you are root. In the output, look for a line looking similar to:
```
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:10050 0.0.0.0:* LISTEN 19843/zabbix_agentd
```
This indicates that the process zabbix_agentd is running and listening on all addresses, on port 10050, just what we need.
Telnetting to the port
Even when a service starts up and successfully binds to a port, there might be some connectivity issues; perhaps due to a local firewall. To quickly check connectivity to the desired port, you can try:
```
$ telnet localhost 10050
```
This command should open a connection to the Zabbix agent daemon, and the daemon should not close the connection immediately.

The Web frontend

Now that we have the Zabbix server and agent compiled, installed, and running, there's probably a feeling that something's missing. We have only configured some low-level behavior, so where's the meat?

That's what the frontend is for. While in theory Zabbix can have multiple frontends, the only one with full functionality so far is web frontend, which was written in PHP. We have to set it up to configure Zabbix and get to those nice graphs everybody likes.

Prerequisites and setting up the environment

Of course, being a web frontend, it will require a platform to run on; a web server with a PHP environment. We will need the following installed:

A web server that is supported by PHP; Apache is the most common choice
PHP version 5

It is easiest to install these from distribution packages. For PHP we'll also need the following functionality:

GD
MySQL
BCmath
mbstring

Note

Some distributions split out core PHP modules. These might include php5-ctype and php-net-socket.

Once you have all these installed, it's time to set up the frontend. First, you have to decide where the frontend code has to go. Most distributions that package web servers use /srv/www/htdocs or /var/www. If you compiled the Apache web server from source, it would be /usr/local/apache2/htdocs (unless you manually changed the prefix or installed an older Apache version). We will place the frontend in a simple subdirectory, zabbix.

Assuming you have distribution packages installed with the web root directory at /srv/www/htdocs, placing the frontend where it is needed is as simple as executing it as root:

# cp -r frontends/php /srv/www/htdocs
# mv /srv/www/htdocs/php /srv/www/htdocs/zabbix

Now it's time to fire up a browser and navigate to Zabbix's address: http://<server_ip_or_name>/zabbix. It should work just fine in the latest versions of most browsers, including Firefox, Opera, Konqueror, and Internet Explorer.

Installation of the web frontend

The web frontend has a wizard that helps you to configure its basics. Let's go through the simple steps it offers.

Step 1 Welcome

If everything is configured properly, you should be greeted by the installation wizard:

If you are not, there are several things that could have gone wrong.

If you see a blank page or some PHP code, make sure that PHP is properly installed and configured to parse files ending with the extension php through the AddType application/x-httpd-php directive. If you see a file and directory listing instead of the installation wizard, make sure you have added index.php to DirectoryIndex directive. If these hints do not help, check the PHP documentation available at http://www.php.net/manual/en/.

This screen doesn't offer us much to configure, so just click Next > >.

Step 2 Licence

On the second screen you are presented with the license, GPLv2. Hopefully you have no objections to this license, so check the I agree box and click Next > > again.

Step 3 PHP prerequisites

In this step, the installation wizard checks PHP-related prerequisites. If you are lucky, all will be satisfied, and you will be greeted with all green entries.

If so, the Next > > button will be enabled, and you just have to click on it to continue to step 4.

Though, more often than not, one or more entries will have a red Fail warning listed next to them. This is where things get more interesting. Problems at this point fall into two categories PHP installation, and configuration.

Entries like PHP version, PHP Databases support, PHP BC math PHP MB string, GD Version, and Image formats are PHP installation problems. To solve these, either install the appropriate distribution packages (sometimes called php5-bcmath, php5-gd, php5-mysql, and so on), or recompile PHP with the corresponding options.

PHP Memory limit, PHP post max size, PHP max execution time, PHP Timezone , and PHP MB string overload are configuration issues that are all set in the php.ini configuration file. This file is usually located at /etc/php5 or similar for distribution packages, and /usr/local/lib for PHP source installations. Set the following options:

memory_limit = 128M
post_max_size = 16M
max_execution_time = 300
mbstring.func_overload = 2

For timezone, set the date.timezone option to a timezone that best matches your environment. For Zabbix home it's Europe/Riga, and you can see valid options at http://www.php.net/manual/en/timezones.php

Note

Since Zabbix version 1.8.2 mbstring.func_overload is not required anymore.

If you can't find php.ini, or you make changes but installation wizard does not pick them up, create a file named test.php in htdocs directory with only this content:

<?php phpinfo() ?>

Navigate using your browser to this file and check the value for entry Configuration File (php.ini) Path - that's where you should look for php.ini.

Once everything is fixed, click Retry button, and you should see more green entries.

Now the Next > > button should be enabled, click on it to continue.

Step 4 Database access.

Remember the database we created before? That's the information we'll insert here. Default database Type, Host, and Port should work for us. Set both Name and User to zabbix. If you have forgotten the password, just look it up or copy it from /etc/zabbix/zabbix_server.conf. After entering the data, click the Test connection button. If all the information is correct, the status should be Ok, so click Next > > again.

Step 5 Zabbix server details

The next screen specifies the Zabbix server location, with defaults being suitable for us, so Next it is again. The following screen is a summary of choices made in the previous screen.

Step 6 Summary

If you left the defaults where appropriate and your database connection test was successful, it should be safe to continue by clicking Next > >.

Step 7 Writing the configuration file

It is quite likely that in the next screen you will be greeted with failure. The installation wizard attempted to save the configuration file, but with the access rights that it has, that should not be possible. There are two possible solutions:

Click Save configuration file and place this file in the htdocs/zabbix/conf directory.

Make a directory htdocs/zabbix/conf writable by web server user (execute as root):

# chown <username> /path/to/htdocs/zabbix/conf
# chmod 700 /path/to/htdocs/zabbix/conf

Obviously, we need to insert the correct username and directory in the above commands. Remember, common locations were /srv/www/htdocs and /usr/local/apache2/htdocs, use the one you copied the Zabbix frontend code to. Common users are wwwrun, www-data, nobody, and daemon you can find out what the correct user for your system is by running:

$ ps aux | grep http

The username that most httpd processes are running under will be the correct one. Once the permissions have been changed, click Retry. That should successfully save the configuration file.

Note

You can also skip configuration wizard by copying zabbix.conf.php.example in the conf directory to zabbix.conf.php and editing it directly.

It is suggested that you restrict the permissions on this file afterwards to be readable only by web server user, by issuing as root:

# chmod 400 /path/to/htdocs/zabbix/conf/zabbix.conf.php
# chown <username> /path/to/htdocs/zabbix/conf/zabbix.conf.php

The file contains the database password which is better kept secret.

Note

It is possible to personalize Zabbix frontend installation a bit. Adding in the zabbix.conf.php configuration file after the line that starts with $ZBX_SERVER_PORT another line $ZBX_SERVER_NAME = "Test server"; will make all pages in Zabbix setup use this name in page titles and display it in the upper-right corner of the frontend, so it would be easier to distinguish this installation from the production instance created later.

Step 8 Configuration file in place

After successfully performing this step and clicking Retry, the installation wizard should be happy, so do as it says and click Next.

Step 9 Finishing the wizard

Congratulations, this is the last wizard screen which only wants you to be friendly to it and press Finish.

Step 10 Logging in

Immediately after clicking Finish you should see a login form.

The Zabbix database data that we inserted previously also supplied the default username and password, so enter admin as the Login name, and zabbix as the Password.

That should get you to the initial frontend screen, which doesn't show much; it only greets you.

Congratulations! The web frontend is now set up and we have logged in.

Note that with some browsers it is possible that you will see either the same page or a blank screen after pressing Finish in the installation wizard, or after logging in. If that is the case, you can try either refreshing the page, or opening the Zabbix root URL, http://<server_ip_or_name>/zabbix.

This view isn't too exciting, so let's check whether the frontend can see if the Zabbix server is running. Hover your mouse over Reports and click Status of Zabbix, the very first report.

So that's some more information already. The frontend successfully sees that Zabbix server is running. It also knows some basic things about Zabbix's configuration there are in total 43 hosts configured. Wait, what's that? We have only set it up and did not configure anything, how can there be 43 hosts already? Let's take a closer look at the Details column. Those values correspond to descriptions in parenthesis, located in Parameter column. So there are 0 monitored hosts, 1 that is not monitored and 42 templates. Now that makes more sense 42 of those 43 are templates, not actual hosts. Still, there's one host that isn't monitored, what's up with that?

Hover your mouse over Configuration and choose Hosts.

So there it is. It turns out that the default Zabbix database already has one server configured; the local Zabbix server. It is disabled by default, as indicated in the Status of Zabbix screen and here by Not monitored string in Status column.