Chapter 7. Splunk SDK for JavaScript and D3.js

In this chapter, we go on to learn about the Splunk Software Development Kit (SDK) and D3.js. Unlike previous chapters, here we will learn some special ways to interact with Splunk, ways that will allow us to create interesting and vibrant applications.

Specifically, in this chapter we will do the following:

• Learn about the Splunk SDK

• Discuss how the SDK can extract data from Splunk

• Find out how a website can be set up to show and use data extracted via the Splunk SDK

• Learn about another important software tool, D3.js, and how it can be used to create useful and impressive data visualizations

We'll begin by talking about Splunk SDKs and how they can be used.

Introduction to Splunk SDKs

A software development kit (also called a SDK or DevKit) is usually a set of software development tools that allows the creation of applications for a certain software package or software framework, but can also refer to a development kit for a computer system, operating system (OS), hardware platform, or even a video game system. We will use the Splunk SDK as a means of extracting data from Splunk and using it for external purposes (for example, a public website).

Splunk actually has several software development kits that sit on top of the REST API. These kits are for Python, Java, JavaScript, PHP, Ruby, and C#, and they allow developers to do all kinds of different things, such as integrating Splunk with third-party tools, logging directly into Splunk, extracting data to create archives, and others. They allow developers to do this using popular programming languages.

These Splunk SDKs do several specific tasks:

1. They handle HTTP access.

2. By utilizing a user ID and...

Practical applications of Splunk's SDK

In this chapter, we will show you how you can extract data from Splunk and display it externally through a web server without the need to log in to Splunk. This is extremely useful when the use case demands a real-time dashboard that is publicly displayed within your organization and that does not require logging in to Splunk.

To achieve this, we will use the Splunk SDK to extract the data from Splunk using a Node.js cron job and dump the payload into JSON files. A cron job involves the cron expressions we learned about in Chapter 5, Data Optimization, Reports, Alerts, and Accelerating Searches, which allow the developer to schedule alerts and other processes very precisely. The term payload is frequently used in computer programming to represent what the person receiving the output from the program is most interested in. The JSON files, which are files based on key-value pairs, will be served by a static web server. We will then create an HTML page...

Creating the final dashboard\jobs.js

Here is the final jobs.js file we will use. Copy this block of code and overwrite the current jobs.js file. We will break this down into functions later and try to explain what the coding does, without putting too much emphasis on the Node.js code itself.

Note

WARNING

When copying from PDF and other files, the encoding may be different, such that it breaks the apostrophes and quotation marks. If you encounter this, just search and replace all apostrophes (') and quotation marks (").

var CronJob = require('cron').CronJob 
var splunkjs = require('splunk-sdk') 
var fs = require('fs') 
 
new CronJob('*/30 * * * * *', function() { 
  // fetch the saved searchName 
  fetchSavedSearch(renderResults, 'sdk_status_codes') 
}, function() {}, true) 
 
var service = new splunkjs.Service({ 
  username:"admin", 
  password:"changed", // Use your own admin password 
  scheme:"https", 
  host:"localhost", &...

Summary

In this chapter, we showed you how to use the Splunk SDK to safely extract data from Splunk using JavaScript without the risk of exposing authentication credentials to viewers. You have also learned how to use Node.js and the npm. Additionally, you wrote a cron script that pulls data from Splunk and writes it into a local JSON file. And you used the http-server module to initialize a lightweight web server. Finally, you created the web application using D3.js to display a stacked bar chart using Splunk data. In the next chapter, Chapter 8, HTTP Event Collector, you will go on to learn about this event collector and how it can be used for many business purposes.