Architecting Splunk

If you have just started learning Splunk, it is unlikely that you would be expected to architect and implement a complex Splunk solution, especially for a larger enterprise. Typically, such projects are advisedly executed with the assistance of experienced architects from Splunk professional services, a Splunk partner consultancy, or your own in-house architects. However, you may be championing an introductory Splunk sandbox or solution at your company, or joining an existing team and need to come up to speed quickly—in which case, this overview should be helpful.

The topics that will be covered in this chapter include the following:

• Collecting the data needed for choosing an appropriate Splunk configuration
• Understanding the different types of Splunk environments
• Understanding replication and search factors
• Introduction to indexing buckets
• Considerations...

Selecting a Splunk configuration

At a high level, the type and size of the Splunk solution you build will depend mostly on two factors:

• The volume of data you will be indexing each day
• The peak number of concurrent searches that will be running

The volume of data indexed will be the average daily total of all the inputs from your various data sources—server logs, network and security devices, and so on. The number of concurrent searches includes both ad hoc searches from users, and the number of scheduled saved searches that will be running periodically to populate reports, dashboards, and alerts. To determine these factors you will need to collect some data from your user communities and do some fairly straightforward calculations.

Data collection – data inputs

...

Selecting Splunk hardware options

If you plan to implement and support your Splunk infrastructure internally (versus leveraging Splunk Cloud, wherein Splunk provides and manages the needed infrastructure for you), you will be building your solution on your own hardware platforms (physical or virtual servers), or on cloud-based instances (AWS, Azure, Google Cloud, and so on). For non-cloud environments, your organization will likely have a preferred hardware vendor that provides support options, so you will be selecting servers within the needed range of CPU, memory, and disk storage options from your vendor's offerings; cloud providers similarly offer a variety of sizing options.

Performance considerations

Splunk provides...

Summary

Whew! We've accomplished a lot of difficult work in this chapter—collecting data from our user groups, selecting the best Splunk configuration to build, learning enough about replication and search factors, indexing buckets, and search head clusters to inform our design decisions, and finally, selecting our hardware options and calculating how much disk space we'll need and how many indexers will be needed to support out expected data ingestion volumes.

After you take a short break, we'll get started installing and configuring our Splunk Enterprise deployment. Don't be long!