Chapter 12. What Next? Splunk 6.4

We already covered various aspects of Splunk 6.3 in the previous chapters in detail. We saw the implementation of various analytics and visualization along with the features of Splunk 6.3. Splunk recently launched an updated version: Splunk 6.4. In this chapter, we will glimpse at all the new features that have been added in Splunk 6.4 to enable better analytics and visualization. Along with the features, we will also see what all changes have been made in Splunk to make it more scalable, functional, and useful to the users. Splunk 6.4, the latest version of Splunk Enterprise comes packed with new features and customizations. The following are the key features that have been added/improved in Splunk 6.4:

• Storage optimization

• Machine learning

• Management and admin

• Indexer and search head enhancement

• Visualizations

• Multi-search management

• Enhanced alert actions

Storage optimization

Splunk 6.4 introduced the new tsidx Retention Policy feature, which allows users to reduce the storage requirements of data available in the cold bucket. The tsidx files are stored under indexers and are responsible for efficient searching in Splunk. Basically, the space taken by historical data available in the cold bucket can be reduced by approximately 50 percent by removing the tsidx indexing information. This can help in saving a lot of money every year that is spent on the storage of old/historical data. This policy can be modified by navigating in the Splunk web interface to Settings | Indexes in Splunk 6.4.

Machine learning

Splunk 6.4 has enhanced the Machine Learning Toolkit and Showcase app, which we already studied with an example in Chapter 5, Advanced Data Analytics. Splunk 6.4 comes with six new machine learning algorithms along with support to hundreds of algorithms of Python's data science library. Apart from this enhancement, the machine learning app has added the Guided ML feature that guides users step by step to build, test, and deploy machine learning models.

Splunk 6.4 has enhanced the predict command with features like these:

• A new algorithm for bivariate time series has been introduced, taking covariance between the individual time series into account for better and efficient prediction

• The predict command can be used to predict results for multiple time series at the same time and can also fill in missing data in the given time series

Management and admin

Splunk 6.4 comes with an enhanced distributed management console, which supports new topology views, search head clustering views, index and storage utilization, and performance views. It also has added support to grant restricted access to admins so that they can manage specific parts of Splunk deployments.

The following are some of the new features added in Splunk 6.4 under the distributed management console:

• The HTTP Event Collector: The management console enlists the entire HTTP Event Collector input classified on the basis of the authorization token. This feature enables the admin to understand and get insight of the data coming in via the HTTP collector input method.

• Search statistics: The console lists the heaviest/long running searches classified on the basis of users. This feature can be used to find out those searches that are causing overhead on Splunk servers.

• I/O statistics: The I/O utilization of bandwidth for Splunk instances is shown to take necessary...

Indexer and search head enhancement

Splunk when deployed in a clustered and distributed environment is now introduced with various enhancements in Splunk 6.4 for higher efficiency and fault tolerance.

The following are the enhancements introduced in Splunk 6.4:

• The index now supports replication of data model and report acceleration summaries. Until Splunk 6.3, if the index failed, the data model and report acceleration summaries were required to be regenerated. In Splunk 6.4, depending on the replication factor, the data model and report acceleration will also be replicated to survive failures.

• In case of overheads or nonperformance of any indexer, the index can be quarantined. This feature restricts any new searches using this indexer, whereas any running searches will continue till the index search gets completed.

• The search head now supports replication of the search peer. This feature enables us to add any nonclustered indexers to a search head cluster. Search head enhancement in Splunk...

Visualizations

Splunk 6.4 has added support of 12 new advanced visualizations directly in the Visualization panel. Some of the new visualizations in Splunk 6.4 were possible in the earlier version of Splunk, that is, Splunk 6.3 using the D3 extension plugin along with the use of customized JS and CSS. Splunk 6.4 adds capability to create any new visualization that can be installed as a plugin directly and share it with other fellow Splunk users.

The following is the list of visualizations introduced in Splunk 6.4 that can be selected and used from the Visualization tab of the Splunk dashboard. When it is said that Splunk 6.4 supports inbuilt visualizations, this means that these visualizations can be directly downloaded from the app store, and the visualizations gets added in the Visualization tab.

The following is the list of visualizations that we have covered in this book in various chapters of visualization using custom CSS and JS. In Splunk 6.4, these visualization apps can be directly...

Multi-search management

We have already seen how the post process was used to enhance the dashboard results based on a global search. Splunk 6.4 has enhanced multi-search management by adding a recursive search post process. Let's understand this enhancement with the help of an example:

Until Splunk 6.3, multi-search management's post process search was based on a global search, that is, a global search is defined and then based on the result of the global search, other post process searches were defined. In the newly enhanced recursive search post process, we can use a search as a base search, which itself is derived from another search. As in the preceding figure, Search 4 is based on the post process of Search 2, where Search 2 itself is based on a post process of a global search.

We have already studied the post process search in this book; now, let's see how to implement the recursive search post process on Splunk 6.4. The following code snippet explains how the recursive post process...

Enhanced alert actions

We already covered custom alert actions in detail in Chapter 9, Advanced Dashboard Customization. In this section, you will learn what new features have been introduced in the Splunk 6.4 release.

Splunk 6.4 has a new feature to choose from the action list of alert actions, that is, it sends log events to the Splunk receiver endpoint. In the following figure, the option marked in the rectangular box is the newly added feature in Splunk 6.4 under alert actions.

This option helps users to redirect the alert log data to Splunk again under the specified sourcetype or index. The alert that used to either trigger e-mails, webhook, or any other defined custom action can also be sent on Splunk for analysis in future. This feature can be helpful for auditing alert scenarios.

Let's understand the use of this Log Event feature in a custom alert. Suppose we have an alert defined to detect fraudulent transactions. Whenever such a transaction is detected, there is a support ticket...

Summary

In this chapter, we had a look at the features and customizations introduced in the latest version of Splunk 6.4. We saw how these features and customizations can be put to use for better use of Splunk's capabilities. In this book, we saw how and where Splunk can be used to make sense out of machine-generated log data and how we can create analytics and visualizations in Splunk. You also learned how to customize dashboards, tweak Splunk, and how to integrate Splunk with analytics and visualization tools.