Chapter 6. Enabling Remote Execution

In this chapter, we will cover the following recipes:

• Monitoring local services on a remote machine with NRPE

• Setting the listening address for NRPE

• Setting allowed client hosts for NRPE

• Creating new NRPE command definitions securely

• Giving limited sudo(8) privileges to NRPE

• Using check_by_ssh with key authentication instead of NRPE

• Using check_mk instead of NRPE

Introduction

For a dedicated Nagios Core server with access to all the relevant parts of the network, making checks is relatively simple using commands and plugins that make ICMP, TCP, and UDP connections to network hosts and services for determining their operating state. These can be used to check any sort of network service, without requiring anything to be installed on the target machine. As an example, when the check_http plugin is used to check a web server, it works the same way as a browser making the request.

However, monitoring a network thoroughly usually has more to it than simply checking network connectivity and availability. It's also a good idea to check properties of the network that don't directly correspond to a network service and hence can't be directly checked over a network connection.

These are often properties of hardware or the underlying system, such as disk space or system load average, or processes that are configured only to listen locally, commonly done for database...

Monitoring local services on a remote machine with NRPE

In this recipe, we'll learn how to install and run an NRPE (Nagios Remote Plugin Executor) server on a target host, roma.example.net. We'll use this to check the load average on that host with the check_load plugin.

The plugins for these checks will be executed on the target server by the NRPE daemon, but the results will be returned to our Nagios Core monitoring server, olympus.example.net. This requires installing the check_nrpe plugin on the monitoring server and the full Nagios Plugins set (but not Nagios Core itself) on the target server.

This is a reasonably long and in-depth recipe as it involves installing a total of three software packages on two servers.

Getting ready

You will need a monitoring server with Nagios Core 4.0 or newer installed. You should also have a UNIX-like target host that you intend to monitor that can run the NRPE daemon. Most modern UNIX-like systems including Linux and BSD should be able to do this. Both...

Setting the listening address for NRPE

In this recipe, we'll learn how to make NRPE listen on a specific IP address on a target host. This might be done on hosts with multiple interfaces in order to prevent spurious requests made to the nrpe daemon from untrusted interfaces, perhaps the public Internet. It could also be appropriate for making the daemon only listen on a trusted VPN interface.

This setup can be particularly useful when the server has an interface into a dedicated management network to which the monitoring server also has access, preventing the nrpe daemon from responding to requests on other interfaces unnecessarily and thereby closing a possible security hole.

Getting ready

You should have a target host configured for checking in a Nagios Core 4.0 or later monitoring server. The target host should be running the nrpe daemon and listening on all interfaces (which we'll fix). You can verify that nrpe is running with pgrep(1) or ps(1):

# pgrep nrpe
29964
# ps -e | grep [n]rpe...

Setting allowed client hosts for NRPE

In this recipe, we'll learn how to configure the nrpe daemon to answer requests from a particular IP address, typically the designated Nagios Core server or servers monitoring your network. This means that nrpe will not run plugins or return results for any check_nrpe request made from IP addresses not in this list.

This is an elementary security step in running an NRPE server, as, if your target host has interfaces or routes into untrusted networks, there is a risk of attackers making spurious requests for information about the system, clogging up your disk with logs from excessive check requests, or even possibly exploiting the nrpe daemon or the Nagios Plugins. This should be done in concert with a hardware or software firewall and security policy.

Getting ready

You should have a target host configured for checking in a Nagios Core 4.0 or later monitoring server. The target host should be running the nrpe daemon. You can verify that nrpe is running with...

Creating new NRPE command definitions securely

In this recipe, we'll learn how to securely create new command definitions for nrpe to run upon request by a monitoring server. We need to do this, because even if we have a huge set of plugins installed on our target host running nrpe, the daemon will only run commands defined in its configuration file.

We'll also learn how arguments can be passed to these commands, if strictly necessary, and about the potentially negative security consequences of this.

Getting ready

You should have a target host configured for checking in a Nagios Core 4.0 or later monitoring server. The target host should be running the nrpe daemon. You can verify that nrpe is running with pgrep(1) or ps(1):

# pgrep nrpe
29964
# ps -e | grep [n]rpe
nagios 29964 1 0 21:55 ? 00:00:01 nrpe

We can inspect the list of commands that nrpe is already configured to run by looking for command directives in its configuration file. By default, this file is /usr/local/nagios/etc/nrpe.cfg...

Giving limited sudo(8) privileges to NRPE

In this recipe, we'll learn how to deal with the difficulty of executing permissions for NRPE. The majority of standard Nagios plugins don't require special privileges to run, although this also depends on how stringent your system's security restrictions are. However, some of the plugins require being run as root or perhaps as another user other than nagios. This is sometimes the case with plugins that need to make requests of system-level resources such as checking the integrity of RAID arrays.

There are four general approaches to fixing this:

• Bad: Change the plugins to setuid, meaning that they will always be run as the user who owns them, no matter who executes them. The problem with this is that setting this bit allows anyone to run the program as root, not just nrpe, a very common vector for exploits.

• Worse: Run nrpe as root or as the appropriate user. This is done by changing the nrpe_user and nrpe_group properties in nrpe.cfg. This is even...

Using check_by_ssh with key authentication instead of NRPE

While all of the previous recipes in this chapter show that NRPE can be very effectively tied down and secured, it may be that we require some means of authentication to a target host in order to run the appropriate Nagios plugins on it. The nrpe daemon does not require any authentication to return information about the host's state; as long as the IP addresses all match and the command is defined for running, it will return information.

If you already use SSH keys for a public key infrastructure in your network, then you may find it preferable to use the check_by_ssh plugin instead, which allows you to use public keys to authenticate with a target host before running any commands. This is only suitable if the target host runs a secure shell daemon.

In this recipe, we'll repeat the setup for the check_load plugin as done in the first recipe in this chapter, Monitoring local services on a remote machine with NRPE, but we'll use the...

Using check_mk instead of NRPE

On very large networks, or networks with a lot of remote checking required, manually configuring NRPE services for everything that requires checking on each host can be burdensome. It's common to automate installing the plugins that are needed on the monitored host and the configuration required to use them, to make the network administrator's job a little easier when adding a set of new hosts with a standard set of services to monitor. Common tools to automate such setups include configuration management tools like Chef, Puppet, or Bcfg2.

Developing standard procedures like this may not appeal and busy administrators may be looking for an extension to Nagios that automatically discovers relevant services on a host to monitor, so that almost no manual configuration is required, instead generating the relevant stanzas automatically. This is one of the main functions of Check_MK, a Nagios Core extension written by Matthias Ketchner and available at https://mathias...