Preface

In a technology-centric world, where machines generate a vast amount of data at an incredibly high volume, Splunk has come up with its industry-leading big data intelligence platform—Splunk Enterprise. This powerful platform enables anyone to turn machine data into actionable and very valuable intelligence.

Splunk Operational Intelligence Cookbook is a collection of recipes that aim to provide you, the reader, with the guidance and practical knowledge to harness the endless features of Splunk Enterprise 6 for the purpose of deriving extremely powerful and valuable operational intelligence from your data.

Using easy-to-follow, step-by-step recipes, this book will teach you how to effectively gather, analyze, and create a report on the operational data available in your environment. The recipes provided will demonstrate methods to expedite the delivery of intelligent reports and empower you to present data in a meaningful way through dashboards and by applying many of the visualizations available in Splunk Enterprise. By the end of this book, you will have built a powerful Operational Intelligence application and applied many of the key features found in the Splunk Enterprise platform.

This book and its easy-to-follow recipes can also be extended to act as a teaching tool for you as you introduce others to the Splunk Enterprise platform and to your new found ability to provide promotion-worthy operational intelligence.

What this book covers

Chapter 1, Play Time – Getting Data In, introduces you to the many ways in which data can be put into Splunk, whether it is by collecting data locally from files and directories, through TCP/UDP port inputs, directly from a Universal Forwarder, or by simply utilizing scripted and modular inputs. You will also be introduced to the datasets that will be referenced throughout this book and learn how to generate samples that can be used to follow each of the recipes as they are written.

Chapter 2, Diving into Data – Search and Report, will provide an introduction to the first set of recipes in this book. Leveraging data now available as a result of the previous chapter, the information and recipes provided here will act as a guide, walking you through searching event data using Splunk's SPL (Search Processing Language); applying field extractions; grouping common events based on field values; and then building basic reports using the table, top, chart, and stats commands.

Chapter 3, Dashboards and Visualizations – Make Data Shine, acts as a guide to building visualizations based on reports that can now be created as a result of the information and recipes provided in the previous chapter. This chapter will empower you to take your data and reports and bring them to life through the powerful visualizations provided by Splunk. The visualizations that are introduced will include single values, charts (bar, pie, line, and area), scatter charts, and gauges.

Chapter 4, Building an Operational Intelligence Application, builds on the understanding of visualizations that you have gained as a result of the previous chapter and introduces the concept of dashboards. The information and recipes provided in this chapter will outline the purpose of dashboards and teach you how to properly utilize dashboards, use the dashboard editor to build a dashboard, build a form to search event data, and much more.

Chapter 5, Extending Intelligence – Data Models and Pivoting, will take you deeper into the data by introducing transactions, subsearching, concurrency, associations, and more advanced search commands. Through the information and recipes provided in this chapter, you will harness the ability to converge data from different sources and understand how to build relationships between differing event data.

Chapter 6, Diving Deeper – Advanced Searching, will introduce the concept of lookups and workflow actions for the purpose of augmenting the data being analyzed. The recipes provided will enable you to apply this core functionality to further enhance your understanding of the data being analyzed.

Chapter 7, Enriching Data – Lookups and Workflows, explains how scheduled or real-time alerts are a key asset to complete operational intelligence and awareness. This chapter will introduce you to the concepts and benefits of proactive alerts, and provide context for when these alerts are best applied. The recipes provided will guide you through creating alerts based on the knowledge gained from previous chapters.

Chapter 8, Being Proactive – Creating Alerts, explains the concept of summary indexing for the purposes of accelerating reports and speeding up the time it takes to unlock business insight. The recipes in this chapter will provide you with a short introduction to common situations where summary indexing can be leveraged to speed up reports or preserve focused statistics over long periods of time.

Chapter 9, Speed Up Intelligence – Data Summarization, introduces two of the newest and most powerful features released as part of Splunk Enterprise Version 6: data models and the Pivot tool. The recipes provided in this chapter will guide you through the concept of building data models and using the Pivot tool to quickly design intelligent reports based on the constructed models.

Chapter 10, Above and Beyond – Customization, Web Framework, REST API, and SDKs, is the final chapter of the book and will introduce you to four very powerful features of Splunk. These features provide the ability to create a very rich and powerful interactive experience with Splunk. The recipes provided will open you up to the possibilities beyond core Splunk Enterprise and a method to make your own Operational Intelligence application that includes powerful D3 visualizations. Beyond this, it will also provide a recipe to query Splunk's REST API and a basic Python application to leverage Splunk's SDK to execute a search.

What you need for this book

To follow along with the recipes provided in this book, you will need an installation of Splunk Enterprise 6 and the sample data that is made available with this book. The recipes are intended to be portable to all Splunk Enterprise environments, but for best results, we suggest that you use the samples provided with this book.

Splunk Enterprise 6 can be downloaded for free for most major platforms from http://www.splunk.com/download.

The samples provided with this book will also be packaged with the Splunk Event Generator tool so that the event data can be refreshed or events can be replayed as new as you work through the recipes.

Who this book is for

This book is intended for all users, beginner or advanced, who are looking to leverage the Splunk Enterprise platform as a valuable Operational Intelligence tool. The recipes provided in this book will appeal to individuals from all facets of a business—IT, security, product, marketing, and many more!

Although the book and its recipes are written so that anyone can follow along, it does progress at a steady pace into concepts or features that might not be common knowledge to a beginner. If there exists the necessity to understand more about a feature, Splunk has produced a vast amount of documentation on all Splunk Enterprise features available at http://docs.splunk.com/Documentation/Splunk.

There might also be sections that utilize regular expressions and introduce recipes that take advantage of the Python and XML languages. Experience with these concepts is not required but beneficial.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "The field values are displayed in a table using the table command."

A block of code is set as follows:

<table>
 <searchString>
  index=opintel status=404 | stats count by src_ip
 </searchString>
 <title>Report – 404 Errors by Source IP</title>

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

<table>
 <searchString>
  index=opintel status=404 | stats count by src_ip
 </searchString>
 <title>Report – 404 Errors by Source IP</title>

Any command-line input or output is written as follows:

./splunk add monitor /var/log/messages –sourcetype linux_messages

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: "Quickly create a report by navigating to Save As | Report above the search bar."

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <[e-mail]>, and mention the book title via the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.

Piracy

Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <[e-mail]> with a link to the suspected pirated material.

We appreciate your help in protecting our authors, and our ability to bring you valuable content.

Questions

You can contact us at <[e-mail]> if you are having a problem with any aspect of the book, and we will do our best to address it.