Reader small image

You're reading from  Splunk Operational Intelligence Cookbook. - Third Edition

Product typeBook
Published inMay 2018
Publisher
ISBN-139781788835237
Edition3rd Edition
Tools
Right arrow
Authors (4):
Yogesh Raheja
Yogesh Raheja
author image
Yogesh Raheja

Yogesh Raheja - - DevOps, Automation and Cloud Coach and Consultant Yogesh Raheja is a certified DevOps and cloud expert with a decade of IT experience. He has expertise in technologies such as OS, source code management, build & release tools, continuous integration/deployment/delivery tools, containers, configuration management tools, monitoring, logging tools, and public/private clouds. He loves to share his technical expertise with audience worldwide at various forums, conferences, webinars, blogs, and LinkedIn. He has written books on IT automation named "Effective DevOps with AWS", "Automation with Puppet 5" and "Automation with Ansible" which has been published by "John & Wiley" and "Packt Publisher". He has also reviewed some of the DevOps books for multiple Publishers.
Read more about Yogesh Raheja

Josh Diakun
Josh Diakun
author image
Josh Diakun

Josh Diakun is an IT operations and security specialist with a focus on creating data-driven operational processes. He has over 10 years of experience managing and architecting enterprise-grade IT environments. For the past 7 years, he has been architecting, deploying and developing on Splunk as the core platform for organizations to gain security and operational intelligence. Josh is a founding partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. He is also a co-founder of the Splunk Toronto User Group.
Read more about Josh Diakun

Paul R. Johnson
Paul R. Johnson
author image
Paul R. Johnson

Paul R. Johnson has over 10 years of data intelligence experience in the areas of information security, operations, and compliance. He is a partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. Paul previously worked for a Fortune 10 company, leading IT risk intelligence initiatives and managing a global Splunk deployment. Paul co-founded the Splunk Toronto User Group and lives and works in Toronto, Canada.
Read more about Paul R. Johnson

Derek Mock
Derek Mock
author image
Derek Mock

Derek Mock is a software developer and big data architect who specializes in IT operations, information security, and cloud technologies. He has 15 years' experience developing and operating large enterprise-grade deployments and SaaS applications. He is a founding partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. For the past 6 years, he has been leveraging Splunk as the core tool to deliver key operational intelligence. Derek is based in Toronto, Canada, and is a co-founder of the Splunk Toronto User Group.
Read more about Derek Mock

View More author details
Right arrow

Data onboarding – defining field extractions

Splunk has many built-in features, including knowledge of several common source types, which lets it automatically know which fields exist within your data. Splunk, by default, also extracts any key-value pairs present within the log data and all the fields within JSON-formatted logs. However, often the fields within raw log data cannot be interpreted out of the box, and this knowledge must be provided to Splunk to make these fields easily searchable.

The sample data that we will be using in subsequent chapters contains data we wish to present as fields to Splunk. Much of the raw log data contains key-value fields that Splunk will extract automatically, but there is one field we need to tell Splunk how to extract, representing the page response time. To do this, we will be adding a custom field extraction, which will tell Splunk how to extract the field for us.

Getting ready

To step through this recipe, you will need a running Splunk server with the operational intelligence sample data loaded. No other prerequisites are required.

How to do it...

Follow these steps to add a custom field extraction for a response:

  1. Log in to your Splunk server.
  2. In the top right-hand corner, click on the Settings menu and then click on the Fields link.
  1. Click on the Field extractions link:
  2. Click on New.
  3. In the Destination app field, select the search app, and in the Name field, enter response. Set the Apply to dropdown to sourcetype and the named field to access_combined. Set the Type dropdown to Inline, and for the Extraction/Transform field, carefully enter the (?i)^(?:[^"]*"){8}s+(?P<response>.+) regex:
  4. Click on Save.
  5. On the Field extractions listing page, find the recently added extraction, and in the Sharing column, click on the Permissions link:
  1. Update the Object should appear in setting to All apps. In the Permissions section, for the Read column, check Everyone, and in the Write column, check admin. Then, click on Save:
  2. Navigate to the Splunk search screen and enter the following search over the Last 60 minutes time range:
index=main sourcetype=access_combined 
  1. You should now see a field called response extracted on the left-hand side of the search screen under the Interesting Fields section.

How it works...

All field extractions are maintained in the props.conf and transforms.conf configuration files. The stanzas in props.conf include an extraction class that leverages regular expressions to extract field names and/or values to be used at search time. The transforms.conf file goes further and can be leveraged for more advanced extractions, such as reusing or sharing extractions over multiple sources, source types, or hosts.

See also

  • The Loading the sample data for this book recipe
  • The Data onboarding – defining event types and tags recipe
Previous PageNext Page
You have been reading a chapter from
Splunk Operational Intelligence Cookbook. - Third Edition
Published in: May 2018Publisher: ISBN-13: 9781788835237
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
undefined
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime

Authors (4)

author image
Yogesh Raheja

Yogesh Raheja - - DevOps, Automation and Cloud Coach and Consultant Yogesh Raheja is a certified DevOps and cloud expert with a decade of IT experience. He has expertise in technologies such as OS, source code management, build & release tools, continuous integration/deployment/delivery tools, containers, configuration management tools, monitoring, logging tools, and public/private clouds. He loves to share his technical expertise with audience worldwide at various forums, conferences, webinars, blogs, and LinkedIn. He has written books on IT automation named "Effective DevOps with AWS", "Automation with Puppet 5" and "Automation with Ansible" which has been published by "John & Wiley" and "Packt Publisher". He has also reviewed some of the DevOps books for multiple Publishers.
Read more about Yogesh Raheja

author image
Josh Diakun

Josh Diakun is an IT operations and security specialist with a focus on creating data-driven operational processes. He has over 10 years of experience managing and architecting enterprise-grade IT environments. For the past 7 years, he has been architecting, deploying and developing on Splunk as the core platform for organizations to gain security and operational intelligence. Josh is a founding partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. He is also a co-founder of the Splunk Toronto User Group.
Read more about Josh Diakun

author image
Paul R. Johnson

Paul R. Johnson has over 10 years of data intelligence experience in the areas of information security, operations, and compliance. He is a partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. Paul previously worked for a Fortune 10 company, leading IT risk intelligence initiatives and managing a global Splunk deployment. Paul co-founded the Splunk Toronto User Group and lives and works in Toronto, Canada.
Read more about Paul R. Johnson

author image
Derek Mock

Derek Mock is a software developer and big data architect who specializes in IT operations, information security, and cloud technologies. He has 15 years' experience developing and operating large enterprise-grade deployments and SaaS applications. He is a founding partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. For the past 6 years, he has been leveraging Splunk as the core tool to deliver key operational intelligence. Derek is based in Toronto, Canada, and is a co-founder of the Splunk Toronto User Group.
Read more about Derek Mock