Reader small image

You're reading from  Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Product typeBook
Published inJan 2013
PublisherPackt
ISBN-139781849693288
Edition1st Edition
Tools
Concepts
Right arrow
Author (1)
VINCENT BUMGARNER
VINCENT BUMGARNER
author image
VINCENT BUMGARNER

Vincent Bumgarner has been designing software for over 20 years, working with many languages on nearly as many platforms. He started using Splunk in 2007 and has enjoyed watching the product evolve over the years. While working for Splunk, he has helped many companies train dozens of users to drive, extend, and administer this extremely flexible product. At least one person in every company he has worked with has asked for a book, and he hopes that this book will help fill their shelves.
Read more about VINCENT BUMGARNER

Right arrow

Reducing summary index size


If the saved search populating a summary index produces too many results, the summary index is less effective at speeding up searches. This usually occurs because one or more of the fields used for grouping has more unique values than is expected.

One common example of a field that can have many unique values is the URL in a web access log. The number of URL values might increase in instances where:

  • The URL contains a session ID

  • The URL contains search terms

  • Hackers are throwing URLs at your site trying to break in

  • Your security team runs tools looking for vulnerabilities

On top of this, multiple URLs can represent exactly the same resource, as follows:

  • /home/index.html

  • /home/

  • /home/index.html?a=b

  • /home/?a=b

We will cover a few approaches to flatten these values. These are just examples and ideas, as your particular case may require a different approach.

Using eval and rex to define grouping fields

One way to tackle this problem is to make up a new field from the URL...

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Implementing Splunk: Big Data Reporting and Development for Operational Intelligence
Published in: Jan 2013Publisher: PacktISBN-13: 9781849693288

Author (1)

author image
VINCENT BUMGARNER

Vincent Bumgarner has been designing software for over 20 years, working with many languages on nearly as many platforms. He started using Splunk in 2007 and has enjoyed watching the product evolve over the years. While working for Splunk, he has helped many companies train dozens of users to drive, extend, and administer this extremely flexible product. At least one person in every company he has worked with has asked for a book, and he hopes that this book will help fill their shelves.
Read more about VINCENT BUMGARNER