Implementing Splunk: Big Data Reporting and Development for Operational Intelligence


Implementing Splunk: Big Data Reporting and Development for Operational Intelligence
eBook: $29.99
Formats: PDF, PacktLib, ePub and Mobi formats
$25.49
save 15%!
Print + free eBook + free PacktLib access to the book: $79.98    Print cover: $49.99
$49.99
save 37%!
Free Shipping!
UK, US, Europe and selected countries in Asia.
Also available on:
Overview
Table of Contents
Author
Support
Sample Chapters
  • Learn to search, dashboard, configure, and deploy Splunk on one machine or thousands
  • Start working with Splunk fast, with a tested set of practical examples and useful advice
  • Step-by-step instructions and examples with a comprehensive coverage for Splunk veterans and newbies alike

Book Details

Language : English
Paperback : 448 pages [ 235mm x 191mm ]
Release Date : January 2013
ISBN : 1849693285
ISBN 13 : 9781849693288
Author(s) : Vincent Bumgarner
Topics and Technologies : All Books, Big Data and Business Intelligence, Data

Table of Contents

Preface
Chapter 1: The Splunk Interface
Chapter 2: Understanding Search
Chapter 3: Tables, Charts, and Fields
Chapter 4: Simple XML Dashboards
Chapter 5: Advanced Search Examples
Chapter 6: Extending Search
Chapter 7: Working with Apps
Chapter 8: Building Advanced Dashboards
Chapter 9: Summary Indexes and CSV Files
Chapter 10: Configuring Splunk
Chapter 11: Advanced Deployments
Chapter 12: Extending Splunk
Index
  • Chapter 1: The Splunk Interface
    • Logging in to Splunk
    • The Home app
    • The top bar
    • Search app
      • Data generator
      • The Summary view
      • Search
      • Actions
      • Timeline
      • The field picker
        • Fields
      • Search results
        • Options
        • Events viewer
    • Using the time picker
    • Using the field picker
    • Using Manager
    • Summary
    • Chapter 2: Understanding Search
      • Using search terms effectively
      • Boolean and grouping operators
      • Clicking to modify your search
        • Event segmentation
        • Field widgets
        • Time
      • Using fields to search
        • Using the field picker
      • Using wildcards efficiently
        • Only trailing wildcards are efficient
        • Wildcards are tested last
        • Supplementing wildcards in fields
      • All about time
        • How Splunk parses time
        • How Splunk stores time
        • How Splunk displays time
        • How time zones are determined and why it matters
        • Different ways to search against time
        • Specifying time in-line in your search
        • _indextime versus _time
      • Making searches faster
      • Sharing results with others
      • Saving searches for reuse
      • Creating alerts from searches
        • Schedule
        • Actions
      • Summary
      • Chapter 3: Tables, Charts, and Fields
        • About the pipe symbol
        • Using top to show common field values
          • Controlling the output of top
        • Using stats to aggregate values
        • Using chart to turn data
        • Using timechart to show values over time
          • timechart options
        • Working with fields
          • A regular expression primer
          • Commands that create fields
            • eval
            • rex
          • Extracting loglevel
            • Using the Extract Fields interface
            • Using rex to prototype a field
            • Using the admin interface to build a field
            • Indexed fields versus extracted fields
        • Summary
        • Chapter 4: Simple XML Dashboards
          • The purpose of dashboards
          • Using wizards to build dashboards
          • Scheduling the generation of dashboards
          • Editing the XML directly
          • UI Examples app
          • Building forms
            • Creating a form from a dashboard
            • Driving multiple panels from one form
            • Post-processing search results
            • Post-processing limitations
              • Panel 1
              • Panel 2
              • Panel 3
              • Final XML
          • Summary
          • Chapter 5: Advanced Search Examples
            • Using subsearches to find loosely related events
              • Subsearch
              • Subsearch caveats
              • Nested subsearches
            • Using transaction
              • Using transaction to determine the session length
              • Calculating the aggregate of transaction statistics
              • Combining subsearches with transaction
            • Determining concurrency
              • Using transaction with concurrency
              • Using concurrency to estimate server load
              • Calculating concurrency with a by clause
            • Calculating events per slice of time
              • Using timechart
              • Calculating average requests per minute
              • Calculating average events per minute, per hour
            • Rebuilding top
            • Summary
            • Chapter 6: Extending Search
              • Using tags to simplify search
              • Using event types to categorize results
              • Using lookups to enrich data
                • Defining a lookup table file
                • Defining a lookup definition
                • Defining an automatic lookup
                • Troubleshooting lookups
              • Using macros to reuse logic
                • Creating a simple macro
                • Creating a macro with arguments
                • Using eval to build a macro
              • Creating workflow actions
                • Running a new search using values from an event
                • Linking to an external site
                • Building a workflow action to show field context
                  • Building the context workflow action
                  • Building the context macro
              • Using external commands
                • Extracting values from XML
                  • xmlkv
                  • XPath
                • Using Google to generate results
              • Summary
              • Chapter 7: Working with Apps
                • Defining an app
                • Included apps
                • Installing apps
                  • Installing apps from Splunkbase
                    • Using Geo Location Lookup Script
                    • Using Google Maps
                  • Installing apps from a file
                • Building your first app
                • Editing navigation
                • Customizing the appearance of your app
                  • Customizing the launcher icon
                  • Using custom CSS
                  • Using custom HTML
                    • Custom HTML in a simple dashboard
                    • Using ServerSideInclude in a complex dashboard
                • Object permissions
                  • How permissions affect navigation
                  • How permissions affect other objects
                  • Correcting permission problems
                • App directory structure
                • Adding your app to Splunkbase
                  • Preparing your app
                    • Confirming sharing settings
                    • Cleaning up our directories
                  • Packaging your app
                  • Uploading your app
                • Summary
                • Chapter 8: Building Advanced Dashboards
                  • Reasons for working with advanced XML
                  • Reasons for not working with advanced XML
                  • Development process
                  • Advanced XML structure
                  • Converting simple XML to advanced XML
                  • Module logic flow
                  • Understanding layoutPanel
                    • Panel placement
                  • Reusing a query
                  • Using intentions
                    • stringreplace
                    • addterm
                  • Creating a custom drilldown
                    • Building a drilldown to a custom query
                    • Building a drilldown to another panel
                    • Building a drilldown to multiple panels using HiddenPostProcess
                  • Third-party add-ons
                    • Google Maps
                    • Sideview Utils
                      • The Sideview Search module
                      • Linking views with Sideview
                      • Sideview URLLoader
                      • Sideview forms
                  • Summary
                  • Chapter 9: Summary Indexes and CSV Files
                    • Understanding summary indexes
                      • Creating a summary index
                    • When to use a summary index
                    • When to not use a summary index
                    • Populating summary indexes with saved searches
                    • Using summary index events in a query
                    • Using sistats, sitop, and sitimechart
                    • How latency affects summary queries
                    • How and when to backfill summary data
                      • Using fill_summary_index.py to backfill
                      • Using collect to produce custom summary indexes
                    • Reducing summary index size
                      • Using eval and rex to define grouping fields
                      • Using a lookup with wildcards
                      • Using event types to group results
                    • Calculating top for a large time frame
                    • Storing raw events in a summary index
                    • Using CSV files to store transient data
                      • Pre-populating a dropdown
                      • Creating a running calculation for a day
                    • Summary
                    • Chapter 10: Configuring Splunk
                      • Locating Splunk configuration files
                      • The structure of a Splunk configuration file
                      • Configuration merging logic
                        • Merging order
                          • Merging order outside of search
                          • Merging order when searching
                        • Configuration merging logic
                          • Configuration merging example 1
                          • Configuration merging example 2
                          • Configuration merging example 3
                          • Configuration merging example 4 (search)
                        • Using btool
                      • An overview of Splunk .conf files
                        • props.conf
                          • Common attributes
                          • Stanza types
                          • Priorities inside a type
                          • Attributes with class
                        • inputs.conf
                          • Common input attributes
                          • Files as inputs
                          • Network inputs
                          • Native Windows inputs
                          • Scripts as inputs
                        • transforms.conf
                          • Creating indexed fields
                          • Modifying metadata fields
                          • Lookup definitions
                          • Using REPORT
                          • Chaining transforms
                          • Dropping events
                        • fields.conf
                        • outputs.conf
                        • indexes.conf
                        • authorize.conf
                        • savedsearches.conf
                        • times.conf
                        • commands.conf
                        • web.conf
                      • User interface resources
                        • Views and navigation
                        • Appserver resources
                        • Metadata
                      • Summary
                      • Chapter 11: Advanced Deployments
                        • Planning your installation
                        • Splunk instance types
                          • Splunk forwarders
                          • Splunk indexer
                          • Splunk search
                        • Common data sources
                          • Monitoring logs on servers
                          • Monitoring logs on a shared drive
                          • Consuming logs in batch
                          • Receiving syslog events
                            • Receiving events directly on the Splunk indexer
                            • Using a native syslog receiver
                            • Receiving syslog with a Splunk forwarder
                          • Consuming logs from a database
                          • Using scripts to gather data
                        • Sizing indexers
                        • Planning redundancy
                          • Indexer load balancing
                          • Understanding typical outages
                        • Working with multiple indexes
                          • Directory structure of an index
                          • When to create more indexes
                            • Testing data
                            • Differing longevity
                            • Differing permissions
                            • Using more indexes to increase performance
                          • The lifecycle of a bucket
                          • Sizing an index
                          • Using volumes to manage multiple indexes
                        • Deploying the Splunk binary
                          • Deploying from a tar file
                          • Deploying using msiexec
                          • Adding a base configuration
                          • Configuring Splunk to launch at boot
                        • Using apps to organize configuration
                          • Separate configurations by purpose
                        • Configuration distribution
                          • Using your own deployment system
                          • Using Splunk deployment server
                            • Step 1 – Deciding where your deployment server will run
                            • Step 2 – Defining your deploymentclient.conf configuration
                            • Step 3 – Defining our machine types and locations
                            • Step 4 – Normalizing our configurations into apps appropriately
                            • Step 5 – Mapping these apps to deployment clients in serverclass.conf
                            • Step 6 – Restarting the deployment server
                            • Step 7 – Installing deploymentclient.conf
                        • Using LDAP for authentication
                        • Using Single Sign On
                        • Load balancers and Splunk
                          • web
                          • splunktcp
                          • deployment server
                        • Multiple search heads
                        • Summary
                        • Chapter 12: Extending Splunk
                          • Writing a scripted input to gather data
                            • Capturing script output with no date
                            • Capturing script output as a single event
                            • Making a long-running scripted input
                          • Using Splunk from the command line
                          • Querying Splunk via REST
                          • Writing commands
                            • When not to write a command
                            • When to write a command
                            • Configuring commands
                            • Adding fields
                            • Manipulating data
                            • Transforming data
                            • Generating data
                          • Writing a scripted lookup to enrich data
                          • Writing an event renderer
                            • Using specific fields
                            • Table of fields based on field value
                            • Pretty print XML
                          • Writing a scripted alert action to process results
                          • Summary

                          Vincent Bumgarner

                          Vincent Bumgarner has been designing software for nearly 20 years, working in many languages on nearly as many platforms. He started using Splunk in 2007 and has enjoyed watching the product evolve over the years. While working for Splunk, he helped many companies, training dozens of users to drive, extend, and administer this extremely flexible product. At least one person at every company he worked with asked for a book on Splunk, and he hopes his effort helps fill their shelves.
                          Sorry, we don't have any reviews for this title yet.

                          Code Downloads

                          Download the code and support files for this book.


                          Submit Errata

                          Please let us know if you have found any errors not listed on this list by completing our errata submission form. Our editors will check them and add them to this list. Thank you.

                          Sample chapters

                          You can view our sample chapters and prefaces of this title on PacktLib or download sample chapters in PDF format.

                          Frequently bought together

                          Implementing Splunk: Big Data Reporting and Development for Operational Intelligence +    Oracle SOA Suite 11g Performance Tuning Cookbook =
                          50% Off
                          the second eBook
                          Price for both: £27.35

                          Buy both these recommended eBooks together and get 50% off the cheapest eBook.

                          What you will learn from this book

                          • How to write searches that are fast and lean
                          • How to create fields from your unstructured data
                          • How to enrich your data with lookups and commands
                          • How to transform your data into useful and beautiful reports
                          • How to build professional looking and informative dashboards
                          • How to make apps to organize and share your searches and dashboards
                          • How to manage configurations for one to thousands of instances
                          • How to integrate with enterprise systems
                          • How to extend Splunk with scripts and advanced configuration

                          In Detail

                          Splunk is a data collection, indexing, and visualization engine for operational intelligence. It's a powerful and versatile search and analysis engine that lets you investigate, troubleshoot, monitor, alert, and report on everything that's happening in your entire IT infrastructure from one location in real time. Splunk collects, indexes, and harnesses all the fast moving machine data generated by our applications, servers, and devices - physical, virtual, and in the cloud.

                          Given a mountain of machine data, this book shows you exactly how to learn to use Splunk to make something useful from it. Depending on your needs, you can learn to search, transform, and display data, or learn to administer your Splunk installation, large or small.

                          "Implementing Splunk: Big Data Reporting and Development for Operational Intelligence" will help you get your job done faster, whether you read from the beginning or jump to what you need to know today. New and experienced users alike will find nuggets of wisdom throughout.

                          This book provides you with valuable examples and step-by-step instructions, showing you how to take advantage of everything Splunk has to offer you, to make the most out of your machine data.

                          "Implementing Splunk: Big Data Reporting and Development for Operational Intelligence" takes you on a journey right from inception to a fully functioning implementation of Splunk. Using a real-world data walkthrough, you’ll be shown how to search effectively, create fields, build dashboards, reports, and package apps, manage your indexes, integrate into the enterprise, and extend Splunk. This practical implementation guide equips you with high-level knowledge for configuring, deploying, extending, and integrating Splunk. Depending on the goal and skills of the reader, enough topics are covered to get you on your way to dashboard guru, app developer, or enterprise administrator. This book uses examples curates reference, and sage advice to help you make the most of this incredibly powerful tool.

                          Approach

                          A step-by-step practical implementation tutorial that equips you with high-level knowledge of configuring, deploying, extending, and integrating Splunk to bring machine-generated operational intelligence (?)to your advantage.

                          Who this book is for

                          The book targets professionals and organizations who want to implement or have already implemented Splunk for log analysis and indexing. Analysts and IT staff for end-to-end investigation, performance monitoring, and so on will also learn from the practical examples. It would even help managers to build reports and summarize the health, performance, and activity of their IT infrastructure and business. You will also find it helpful as a technical administrator, consultant, or end user.

                          This book aims to be useful to Splunk users of all levels, from complete newbie to seasoned user. The book assumes that you have access to a copy of Splunk, ideally not in production. Many examples also assume your user has admin rights.

                          Code Download and Errata
                          Packt Anytime, Anywhere
                          Register Books
                          Print Upgrades
                          eBook Downloads
                          Video Support
                          Contact Us
                          Awards Voting Nominations Previous Winners
                          Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
                          Resources
                          Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software