Chapter 5. Data Management

Retrieving data for graphing with Cacti is more than just pulling SNMP data. Cacti allows several different methods for data retrieval. Besides SNMP, Cacti is able to retrieve data using remotely executed scripts, pulling data from databases, or building data by utilizing the Windows Management Interface (WMI).

In this chapter we are going to:

• Provide a short overview of Cacti's data management

• Explain how to build a data input method

• Build a data query

• Develop a complete remote SSH data input method

Let's get started!

An introduction to Cacti data management

Cacti provides different methods for remote data query. You have already learned about the data input methods for retrieving a single SNMP value, but this method is not limited to SNMP. You can use several other methods such as scripts or the WMI for retrieving and graphing data.

Data input methods

Data input methods are commonly used when it comes to basic data retrieval. They can be external scripts, simple SNMP queries, or anything that does not involve indexed data.

Data queries

Data queries are used for indexed data. A good example data is retrieving interface statistics for switches where each data entry (for example, for interface inOctets) is linked to the interface using an index. So, for complex data, you will need to create a data query.

Creating data input methods

We have already seen how to create a data input method for simple SNMP queries. Let's look into creating a script-based data input method for retrieving some information out of the secure log file. The secure log file contains information about failed and successful logins. Please note that the following example will only work on a Linux system.

Preparation—creating the script

For a script/command data input method, you obviously need a script for gathering the required data and providing it to Cacti. Look at the unix_secure_log.pl file. This Perl script will collect the necessary data for you.

Let's have a look at the commands used in the script. Within the first line of interest, you configure the path and filename to look for the data:

# Where do we find the secure log file ?
my $secure_log = '/var/log/secure';

Gathering the data

The following important lines within the script retrieve the relevant Cacti data by executing a sequence of operating system commands:

...

Time for action – creating a data input method – Step 1

Logon to your Cacti installation as an admin user and change to the Console tab.

1. Go to Collection Methods | Data Input Methods.

2. Click on the Add link at the top right.

3. Enter Unix - Secure Log Input as the Name.

4. Select Script/Command as the Input Type .

5. Enter perl <path_cacti>/scripts/unix_secure_log.pl as the Input String.

6. Click on the Save button.

   

What just happened?

You just created a basic script/command-based data input method. You also told Cacti how to execute the script by defining the Input String. Selecting the Script/Command as Input Type made Cacti automatically add fields for data input and data output after clicking on the Save button.

Time for action – creating a data input method – Step 2

1. Click on the Add link at the Output Fields table. This will open a new form.

2. Enter InvalidUsers as the Field [Output].

3. Enter InvalidUsers as the Friendly Name.

4. Click on the Create button.

5. Click on the Add link at the Output Fields table again.

6. Enter ValidLogons as the Field [Output].

7. Enter ValidLogons as the Friendly Name.

8. Click on the Create button.

   

What just happened?

You just added the two output fields from the script to this data input method. Cacti is now able to use this information and fill the RRD files with the data retrieved from the script.

The data template

As you've already learned in Chapter 3, Creating and Using Templates, you will need to create a data template for this data input method to be used within a graph. You can now either import the template that comes with this book, or better still, try to create the data template yourself. The following picture shows the final data template:

Creating data queries

Data queries are more complex than simple data input methods. As already mentioned, they involve indexes. In order for you to get off to an easy start with data queries, you will now rebuild the SNMP-based data query for network interfaces.

Building the XML data file

Building a script or SNMP query always starts with an XML file providing the relationship between input, output, and the index. Look at the interface_book.xml file that comes with this book for an example of such an XML file.

XML header

The XML header sets some generic information for Cacti. A generic description as well as the SNMP OID index can be set here:

<name>Get SNMP Interfaces</name>
<description>Queries a host for a list of monitorable interfaces</description>
<oid_index>.1.3.6.1.2.1.2.2.1.1</oid_index>
<oid_num_indexes>.1.3.6.1.2.1.2.1.0</oid_num_indexes>
<index_order>ifDescr:ifName:ifHwAddr:ifIndex</index_order>
<index_order_type>numeric...

Time for action – creating a data query – Step 1

1. Go to Collection Methods | Data Queries and click on the Add link which is to the top right.

2. Enter SNMP - Interface Statistics Simple as the Name.

3. Enter Simple ( In/OutOctets ) Interface Statistics as the Description.

4. Enter <path_cacti>/resource/snmp_queries/interface_book.xml as the XML Path.

5. Select Get SNMP Data (Indexed) as Data Input Method.

6. Click on the Save button.

   

What just happened?

You just told Cacti where to find the XML file which you created earlier and what data input method to use for retrieving the data. You can now add one or more graph templates to this query.

During the next step you're going to add a graph and a data template to this data query. This step is needed as a data query can provide different "views" on the data. Interface traffic, for example, can consist of 32bit or 64bit values, or you may want to have the total bandwidth usage displayed, as calculated on some graphs. As all of this data, as well as the graphs...

Time for action – creating a data query – Step 2

Now you're going to add one Associated Graph Template to this data query.

1. Click on the Add link on the Associated Graph Templates table

2. Enter In/Out Bits as the Name.

3. Select Interface – Traffic (bits/sec) as the Graph Template. If you do not find the graph template you need, you will need to create one first.

4. Click on the Create button. Several new tables will appear now:

   

5. Select ifOutOctets (Bytes Out) for the traffic_out data source.

6. Check both checkboxes from the Associated Data Templates table.

7. Enter |host_description|- Traffic - |query_ifName| in the first field of the Data Template – Interface –Traffic section.

8. Enter name as the Field Name and click the Add button next to it.

9. Enter |query_ifSpeed| in the first field of the Data Template – Interface –Traffic section.

10. Enter rrd_maximum as the Field Name and click on the Add button again. You will now have two entries in the first table:

    

11. Now enter |host_description|- Traffic - |query_ifName| in the...

Create a remote SSH data input method

Besides SNMP queries, remote data retrieval is an important part of creating Cacti performance graphs. You may get into a situation where SNMP data retrieval is not possible and you only have SSH access to a device. Let's assume you want to graph the I/O performance of some server drives using the iostat utility. You will now learn the different steps it takes to create such a remote SSH data input method.

Preparation

As a preparation for the data input method, you will need to create two different scripts and set up public key authentication.

The remote script

The remote script will execute local commands and preparse/prepare the data on the remote system. The data will be returned to the local script. If you have not already installed the iostat utility on the remote system, you can call the following command to do so on a CentOS system:

yum install sysstat

This will install the iostat utility on the system.

Let's call this utility and look at the output...

Time for action – create SSH keys with PuTTY Key Generator

1. Start the PuTTY Key Generator .

2. Click on the Generate button and follow the instructions provided.

   

3. Click on the Conversions menu, and then click on the Export OpenSSH key entry.

4. Store the key without a passphrase somewhere on your local disk.

5. Open the file using Notepad.

6. Copy the whole content of the file.

7. Connect to your Cacti system using PuTTY.

8. Issue the following command:

   vi /home/cactiuser/ioStats_keyfile
   

9. Press the i key to get into the Insert mode.

10. Right click on the putty screen to insert the content of your clipboard to the file. This should still be the content of the PuTTY Key Generator.

11. Save the file and exit by pressing ESC and then entering :x.

12. Execute the following command to make the file only readable by your Cacti user:

    chown cactiuser /home/cactiuser/ioStats_keyfile
    chmod 600 /home/cactiuser/ioStats_keyfile
    

13. Go back to your PuTTY Key Generator window and copy the text from the upper part (Public key).

14. Connect to your remote...

Time for action – creating the data input method

1. Create a data input method.

2. Enter Unix - Remote IOStats Command as the Name.

3. Select Script/Command as Input Type.

4. Enter perl <path_cacti>/scripts/diskIo.pl <host> <user> <sshKey> <diskDevice> as the Input String.

5. Click on the Create button.

6. Click on the Add link at the Input Fields section.

7. Select host as the Field [input].

8. Enter Hostname as the Friendly Name.

9. Enter hostname as the Special Type Code.

10. Click on the Create button.

11. Add the other input fields by only adding a Friendly Name.

12. Add the output fields for Reads,Writes,ReadsKb,WritesKb, and ioUtil.

    

What just happened?

You've just created the data input method for retrieving the output from the iostats command of a remote system. You've defined the input fields such as the remote username to use for logging in to the system, as well as the location of the SSH key which you defined earlier. The data retrieved from this command will be stored in the output fields which...

Before we continue

Creating data input methods using scripts or queries involves quite some programming and network knowledge. This chapter built up the complexity by going from a simple local data input method to a more complex data query and building a remote executed input method. If you feel lost in any of these exercises you should try to focus on one of them before moving on to the next step. If you want to learn more about data queries, you should visit the HowTo sections on the Cacti website, as well as the relevant section in the Cacti online manual.

• HowTo: http://docs.cacti.net/howto:data_query_templates.

• Manual: http://docs.cacti.net/manual:088:3a_advanced_topics.3d_script_data_query_walkthrough.

Pop quiz – a few questions about Chapter 5

1. Where will you define the path to the SSH key file?

   a. In the config.php file of Cacti

   b. In the Cacti settings page

   c. When adding the special graph template to a device

   d. When adding or editing a device

2. What do you need an indexed SNMP query for?

   a...

Summary

You should now be able to create your own data input methods and create custom scripts to gather remote data.

This chapter covered:

• Creating data input methods—how to gather data from the local Cacti system using a local script

• Creating data queries—creating the underlying XML file and gathering indexed data for displaying in Cacti

• The remote SSH data input method—how to retrieve and display data from a remote system using the SSH public key authentication method

Having covered the data management chapter, you should now be able to comfortably manage a Cacti system. In the next chapter, you're going to take a closer look into maintaining Cacti by scheduling backup jobs, maintaining log files, and more.