Reader small image

You're reading from  Advanced Splunk

Product typeBook
Published inJun 2016
Publisher
ISBN-139781785884351
Edition1st Edition
Tools
Right arrow
Author (1)
Ashish Kumar Tulsiram Yadav
Ashish Kumar Tulsiram Yadav
author image
Ashish Kumar Tulsiram Yadav

Ashish Kumar Tulsiram Yadav is a BE in computers and has around four and a half years of experience in software development, data analytics, and information security, and around four years of experience in Splunk application development and administration. He has experience of creating Splunk applications and add-ons, managing Splunk deployments, machine learning using R and Python, and analytics and visualization using various tools, such as Tableau and QlikView. He is currently working with the information security operations team, handling the Splunk Enterprise security and cyber security of the organization. He has worked as a senior software engineer at Larsen & Toubro Technology Services in the telecom consumer electronics and semicon unit providing data analytics on a wide variety of domains, such as mobile devices, telecom infrastructure, embedded devices, Internet of Things (IOT), Machine to Machine (M2M), entertainment devices, and network and storage devices. He has also worked in the area of information, network, and cyber security in his previous organization. He has experience in OMA LWM2M for device management and remote monitoring of IOT and M2M devices and is well versed in big data and the Hadoop ecosystem. He is a passionate ethical hacker, security enthusiast, and Linux expert and has knowledge of Python, R, .NET, HTML5, CSS, and the C language. He is an avid blogger and writes about ethical hacking and cyber security on his blogs in his free time. He is a gadget freak and keeps on writing reviews on various gadgets he owns. He has participated in and has been a winner of hackathons, technical paper presentations, white papers, and so on.
Read more about Ashish Kumar Tulsiram Yadav

Right arrow

Chapter 11. Enterprise Integration with Splunk

We now have enough understanding of how to use Splunk for analytics and visualization. In this chapter, we will go through how Splunk can be integrated with any present/legacy proprietary applications in detail along with examples. Splunk provides an Software Development Kit (SDK) on almost all programming languages, such as .NET, Java, Python, and so on. The SDK can be used to integrate with applications to get better, efficient, and faster (real-time) results in the applications. You will also learn how Splunk can be integrated with other tools such as R for analytics and Tableau for visualization.

The following are the topics that will be covered in this chapter:

  • The Splunk SDK

  • Installing the Splunk SDK

  • The Splunk SDK for Python

  • Splunk with R for analytics

  • Splunk with Tableau for visualization

The Splunk SDK


An SDK plays a very important role in integrating the power of Splunk's real-time analytics and visualization in legacy/proprietary applications. Industries and organizations use some or the other tool to generate analytics and visualization. However, legacy/propriety tools may not be scalable to handle big data and provide real-time analytics, and hence Splunk comes to the rescue. It may not be possible to replace the current tools used in the workflow, and hence, the Splunk SDK can be used to integrate with the current tool to utilize the power of Splunk.

The Splunk SDK is available in almost all the programming languages, such as C#, Java, PHP, Ruby, Python, and JavaScript.

The following are the scenarios where the Splunk SDK can be useful:

  • It can be integrated with current workflow tools seamlessly to utilize the power of Splunk's big data analytics and visualization in real time.

  • An SDK can help in logging data in the Splunk server directly from the application; that is,...

Installing the Splunk SDK


We already know that the Splunk SDK is available for most of the popular programming languages, but we will concentrate on the Splunk SDK for Python on a Windows OS in detail in this chapter.

The Splunk SDK is available for download from the Splunk website (http://dev.splunk.com/sdks) and the Splunk SDK for Python can be directly downloaded from http://dev.splunk.com/goto/sdk-python.

The SDK is for Python, and hence, Python should be already installed to use the Splunk SDK for Python. According to Splunk documentation, the Splunk SDK supports Python 2.6 or higher but does not support Python 3.

The Python SDK can be installed in Windows using Python's Setuptools or by downloading the Python SDK from the preceding link manually. The Python SDK can be installed from the Command Prompt as follows:

  1. Download Setuptools from the Python website (https://pypi.python.org/pypi/setuptools) and in Command Prompt, navigate to the Setuptools folder and type Python easy_install.py...

The Splunk SDK for Python


We understood the use of an SDK and also saw the installation part of the Splunk SDK for Python. Now, we will see how the SDK can be used to integrate the power of Splunk's analytics and visualization.

Importing the Splunk API in Python

The following import statement will make the Splunk API available for use in Python:

import splunklib.client as client
import splunklib.results as results

The preceding two import statements make the API exposed using the Splunk SDK that is to be used in the Python code for integration.

Connecting and authenticating the Splunk server

The following image in the code snippet connects and also authenticates the Splunk server. The login details can be passed as a parameter when running the Python code or can be hardcoded in the code itself and can be saved in a .splunkrc file:

Once the authentication is successful, the Splunk APIs can be used to send data on Splunk, enlist or run saved searches, run a search query on Splunk, upload files...

Splunk with R for analytics


We now have enough knowledge of Splunk's features and analytical capabilities; let's look at R and its capabilities. R is a statistically and graphically supported programming language for data analysis and data mining. R has extensive library support for statistical computing (linear/nonlinear modeling, clustering, classification, time series analysis, graphical plotting, predicting, forecasting, data mining, and so on).

Splunk, being a big data tool, can be integrated with R to leverage its advanced analytical capabilities for real-time insights. The Splunk app store had an app called R Project, but it is no longer available on the app store. The R Project app for Splunk can now be downloaded from GitHub (https://github.com/rfsp/r/).

The app can be installed on a Splunk instance like any other app downloaded from the Splunk app store. This app on Splunk exposes a new search command—r, which allows us to pass data from Splunk to the R-Engine for calculation and...

Splunk with Tableau for visualization


In the preceding section, we saw how to use R along with Splunk to generate useful insight from the data using R libraries from Splunk itself. In this section, we will see how to use the processing power of Splunk and the visualization power of Tableau for interactive visualization.

Tableau is a very advanced, interactive, business-intelligent software. It helps in deriving instantaneous insights by data transformation into interactive visualizations. It has an easy-to-use drag and drop feature that helps in making highly useful dashboards in minutes. Splunk already has a pivot feature and ample visualization, but Tableau can be used to answer many unknown questions from the data.

With Splunk being a big data tool and Tableau being an excellent interactive visualization tool, their integration can be very useful to derive insight and take informed business decisions on time.

The setup

There are various variants of Tableau, and a suitable version of Tableau...

Summary


In this chapter, you learned about the Splunk SDK, its setup procedure, uses, and enterprise integration of Splunk using Python. We also had a look at how Splunk can be used to integrate with R for analytics and Tableau for visualization. In the next chapter, we will look at the features introduced in the latest version of Splunk 6.4 along with examples and their uses.

lock icon
The rest of the chapter is locked
You have been reading a chapter from
Advanced Splunk
Published in: Jun 2016Publisher: ISBN-13: 9781785884351
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
undefined
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime

Author (1)

author image
Ashish Kumar Tulsiram Yadav

Ashish Kumar Tulsiram Yadav is a BE in computers and has around four and a half years of experience in software development, data analytics, and information security, and around four years of experience in Splunk application development and administration. He has experience of creating Splunk applications and add-ons, managing Splunk deployments, machine learning using R and Python, and analytics and visualization using various tools, such as Tableau and QlikView. He is currently working with the information security operations team, handling the Splunk Enterprise security and cyber security of the organization. He has worked as a senior software engineer at Larsen & Toubro Technology Services in the telecom consumer electronics and semicon unit providing data analytics on a wide variety of domains, such as mobile devices, telecom infrastructure, embedded devices, Internet of Things (IOT), Machine to Machine (M2M), entertainment devices, and network and storage devices. He has also worked in the area of information, network, and cyber security in his previous organization. He has experience in OMA LWM2M for device management and remote monitoring of IOT and M2M devices and is well versed in big data and the Hadoop ecosystem. He is a passionate ethical hacker, security enthusiast, and Linux expert and has knowledge of Python, R, .NET, HTML5, CSS, and the C language. He is an avid blogger and writes about ethical hacking and cyber security on his blogs in his free time. He is a gadget freak and keeps on writing reviews on various gadgets he owns. He has participated in and has been a winner of hackathons, technical paper presentations, white papers, and so on.
Read more about Ashish Kumar Tulsiram Yadav