Splunk has many built-in features, including knowledge on several common source types, which lets it automatically know which fields exist within your data. Splunk, by default, also extracts any key-value pairs present within the log data and all the fields within the JSON-formatted logs. However, often the fields within raw log data cannot be interpreted out of the box, and this knowledge must be provided to Splunk in order to make these fields easily searchable.
The sample data that we will be using in subsequent chapters contains data we wish to present as fields to Splunk. Much of the raw log data contains key-value fields that Splunk will extract automatically, but there is one field we need to tell Splunk how to extract, representing the page response time. To do this, we will be adding a custom field extraction, which will tell Splunk how to extract the field for us.
To step through this recipe, you will need a running Splunk server with the operational intelligence sample data loaded. No other prerequisites are required.
Follow the given steps to add a custom field extraction for response:
Log in to your Splunk server.
In the top right-hand corner, click on the Settings menu and then click on the Fields link.
Click on the Field extractions link.
Click on New.
In the Destination app field, select the search app, and in the Name field, enter
response. Set the Apply to drop-down list to sourcetype and the named field toaccess_combined. Set the Type drop-down list to Inline, and for the Extraction/Transform field, carefully enter the(?i)^(?:[^"]*"){8}\s+(?P<response>.+)regex.
Click on Save.
On the Field extractions listing page, find the recently added extraction, and in the Sharing column, click on the Permissions link.
Update the Object should appear in setting to All apps. In the Permissions section, for the Read column, check Everyone, and in the Write column, check admin. Then, click on Save.
Navigate to the Splunk search screen and enter the following search over the Last 60 minutes time range:
index=main sourcetype=access_combined
You should now see a field called response extracted on the left-hand side of the search screen under the Interesting Fields section.
All field extractions are maintained in the props.conf and transforms.conf configuration files. The stanzas in props.conf include an extraction class that leverages regular expressions to extract field names and/or values to be used at search time. The transforms.conf file goes further and can be leveraged for more advanced extractions, such as reusing or sharing extractions over multiple sources, source types, or hosts.
