Chapter 4. Data Analytics

This chapter will help you understand how to analyze the data and get insight on the data that is uploaded on Splunk from various sources. Right from searching, sending search results over e-mail, combining search results, and accessing the data, you will be able to do basic analytics and data manipulation on Splunk Enterprise via the web console. The reader will also be able to add, extract, and modify fields and format the output as per their requirements. We will use Splunk search commands to fetch the desired the insights and statistics on Splunk Enterprise.

In this chapter, we will cover the following topics:

• Data and indexes

• Search

• Subsearch

• Time

• Fields

• Results

Data and indexes

When data is sent on Splunk Enterprise, it consumes the raw data and converts it into searchable events. This processed data gets stored in an index in Splunk. We will now go through the search commands that can be used on Splunk Web to view and manage the data and indexes.

You will now learn to use Splunk commands to analyze the data. There are many Splunk commands, and each command has many parameters. We will go through the important commands and the required parameters.

Accessing data

The following set of commands can be used to access data from indexes. These categories of commands just fetch the information and display it. They do not modify the data or indexes.

The index command

Splunk's index command displays the event data of the specified index. On installation, Splunk Enterprise already has the default index as main. It also has few other indexes names, such as _audit, _internal, _introspection, and so on. They can be used for Splunk error lookup, Splunk health status...

Search

Splunk is said to be the Google of machine data. So, searching is the most important set of actions that is performed to retrieve the exact information the user is looking for from the indexes. You will now learn how to make efficient use of search commands to fetch the relevant and required information precisely from the whole set of data.

The search command

The search command is used to search events and filter the result from the indexes. The search command, followed by keywords, phrases, regular expressions, wildcards, and key-value pairs, can be used to fetch filtered events from the indexes.

Mentioned as follows is the syntax for a search command instance:

<keywords>
    <wildcards>
    <key_value_pairs> or <fields>
    <phrases>
    <operators>
    <logical_expressions>
    <regular_expressions>
    <time_specifiers>

The parameter description for the preceding parameters is as follows:

• keywords: A keyword can be any string...

Subsearch

The search that is enclosed in a square bracket and whose result is passed as a parameter value to the search is called a subsearch. Basically, subsearches are used when the search requires some input that cannot be directly specified or that keeps on changing. Hence, another search query is written, and the result is passed to the original search.

Let's assume a user wants to know the location and IP address of top three users who have failed the login attempt. Now, the top three users who are failing the login will keep on changing, so subsearches are used. The subsearch will show the top three users that will be passed to the original search. This search will result in the location and IP address of those three users.

You will learn in detail how the preceding scenarios can be solved using various subsearch commands.

The append command

The append command of the subsearch category, as the name suggests, is used to append the result of one search with another search result. The subsearch...

Time

The time subset of commands is used to enrich the data with the ability to search based on time and make data more user friendly for analytics and visualization.

The reltime command

The reltime Splunk command is used to create a relative time field called reltime. It shows the time value in a format that humans can read, relative to current time. The time in reltime would appear as 2 hours ago, 3 days ago, 1 month ago, and so on.

The syntax for the reltime command is as follows:

… | reltime

Refer to the following example for better clarity:

index=_internal |reltime

As shown in the preceding screenshot, reltime creates a more user friendly and human readable format output of relative time, which can be used in analytics and visualizations.

The localize command

The localize command is used to create a list of time ranges in which the results have occurred.

The syntax for the localize command is as follows:

localize maxpause

The maxpause parameter can be used to specify the maximum time between...

Fields

The fields subset of commands on Splunk is used to add, extract, and modify fields and field values. These commands help users enrich the data, do mathematical and string operations on the fields, and derive insight from the data.

The eval command

The eval command of Splunk is very useful and powerful. It can be used to evaluate Boolean, mathematical, or string expressions. It can also be used to create custom (new) fields using existing fields or arbitrary expressions. This command can be used to create new fields, which is the result of some calculations, or use conditional operators such as if, case, match, and so on to apply some expression and evaluate the result.

The eval command can also be used to coalesce fields from different sources or indexes to create a transaction of events. The command can be used to extract information to create multiple fields from an event(s).

The syntax for the eval command is as follows:

eval fieldname = expression/Functions

The parameter description...

Results

The Results set of commands is used to manage the output of the search results. This set of commands can be used to filter the events, reformat the events, group them, reorder them, and read and write on the results.

The fields command

The fields command is used to keep (+) or remove (-) fields from the search results. If + is used, then only the field list followed by + will be displayed, and if – is used, then the field list followed by – will be removed from the current result set.

The syntax for the fields command is as follows:

… | fields +/- field_list

Refer to the following example for better clarity:

index=_internal | top component cumulative_hits executes | fields – percent

In the preceding screenshot, we have used the top command. The top command returns the count and percentage of the specified fields. So, we have used fields – percent, which shows all the fields, except percent. Similarly, the fields command can be used to get the desired output.

The searchtxn command

The...

Summary

In this chapter, you studied Visualization using examples and gained in-depth knowledge on using the important search commands needed for data manipulation, searching, and basic analysis on the uploaded data. Now, in the next chapter, you will study with practical examples and illustrations, how to use advanced analytics commands.