All that being said, let's talk Splunk applications. A Splunk application is nothing more than a structured set of configurations and assets used to achieve the end goal of data collection, indexing, and visualization. Furthermore, in order to create a valid Splunk application, you must have included the ability to navigate. Without navigation within the application, you are now working on an add-on. According to Splunk, applications:
Contain at least one navigable view
Can be opened from the Splunk Enterprise home page, from the App menu, or from the Apps section of Settings
Focus on aspects of your data
Are built around use cases
Support diverse user groups and roles
Run in tandem
Contain any number of configurations and knowledge objects
Are completely customizable, from frontend to backend
Can include web assets such as HTML, CSS, and JavaScript
This is available at: http://docs.splunk.com/Documentation/Splunk/6.2.2/AdvancedDev/AppIntro
Applications allow us to quickly share configurations, limit data access, or organize similar dashboards and views into a cohesive presentation of data within Splunk. Sharing applications can be as easy as just zipping it up and sending it out. Splunk applications could be said to be open source, due to the fact that almost all of the configurations, custom scripts, and any other any other knowledge object contained within the applications, are readable on the file system. This allows for customization for an individual instance while maintaining an overall "master configuration".
To get started, we should define a few "naming conventions" typically used when naming applications. Note that while we will use these naming conventions as the best practice, your application can really be named anything at all, which may conflict with other applications of the same name, or violate Splunk usage agreements or publishing guidelines. In particular, the word "Splunk" cannot be present in your application or add-on name. Additionally, in the past, Splunk has referred to add-ons as "technology add-ons", and has since moved to just "add-ons". The following list of add-on types is our way to distinguish the different uses of each add-on:
Tip
Follow the Splunk application design guidelines. Using a custom naming scheme may cause conflicts.
Applications: Applications could be named anything, as long as they are relevant to the content of the application and don't contain the word "Splunk".
Domain add-ons (DA): Domain add-ons are not full applications, rather they contain the visualizations and presentation of the data for a broader application. No other configurations should be included (extracts, tags, event types, macros, line breaking configurations, and so on). Dashboards and views are the primary objects contained within this type of add-on.
Supporting add-ons (SA): Supporting add-ons are also not full applications—these contain "data definitions", such as macros, saved searches, event types, and tags. These describe how to correlate the data, normalize the data, and consolidate the data to be usable in the domain add-on.
Technical add-ons (TA): Technical add-ons provide extraction, data massage, and index-time configurations. These contain the configuration options required to properly break events, extract search fields, and timestamps (among other functions). These are the building blocks for the SA and DA add-ons, as well as full-blown applications.
Thus, end the "official" naming conventions as normally seen in a Splunk installation. We will now discuss some other naming conventions that have been found to help in the Wild West of various Splunk installations. These two naming conventions are of the author's own design, which have helped in some of his deployments:
Input add-on (IA): Input add-ons are just that—configurations that assist in the collection of data, known as inputs. These add-ons are most likely found on a deployment server and are used to collect data from universal forwarders. One of the advantages of splitting your IAs from your TAs is a reduced size in the add-on being sent to the universal forwarder. This is especially useful if your TA contains lookups that aren't needed on the universal forwarder but are several megabytes in size.
Admin add-on (ADMIN): This add-on is a very special add-on. It would typically contain "administrative" configurations that might be needed in a variety of locations. Such configurations could be the web server SSL port, deployment client information, or anything in
web.conf
orserver.conf
format. It can be used to send index information to a set of non-clustered indexers, or possibly to scale the addition of more search heads by setting all relevant settings from a central location.
While this may not be a complete list of naming conventions, it should be enough to recognize any that are seen in the wild. An additional aspect of the naming conventions that we recommend you take note of is the addition of company information. This will help your Splunk admins differentiate between Splunk add-ons and custom add-ons. Just as an example, let's say that you built a TA for Cisco, specific to your company (the ACME company). Splunk's provided add-on is entitled "TA-cisco", but you don't want to modify a vendor's offering. So, your new add-on's name could be "A-ACME-TA-cisco". This gives you two things:
An easy-to-see custom TA that relates to Cisco
The ability to override any TA-cisco settings based on application precedence
Let's discuss application precedence for a moment. Splunk uses a "merged configuration" when applying configurations that are installed via the applications. The methodology that Splunk chose to implement conflict resolution is pretty simple. There are two different methods of precedence.
The first is directory structure. If you have an input located in the default
folder of an application (more on default
in following chapters), you can place a matching configuration in the local
folder of the application to override the default
configuration. The same method is applied to the applications themselves. Splunk uses the ASCII values of the names to determine precedence. On *nix, you can sort the applications in the apps
folder of Splunk using the LC_COLLATE=C ls
command. This will show you the ASCII sorted order of the Apps, and the first in the list will be the highest priority. A has a higher priority over Z, but Z has a higher priority over a. So, the A at the beginning of the add-on name gives your add-on the highest precedence, so you can override any setting as needed.