Manipulating raw data (pre-indexing)

There are some advantages and disadvantages to using a Heavy Forwarder to deliver cooked data to the indexer. The primary advantage this gives us is better performance at search time. If we extract fields and break events before we write to disk, then it's less resource-intensive on the search head, and the indexer at search time.

The primary disadvantage only comes when we do these instances on the indexer itself, instead of offloading these types of operation to a Heavy Forwarder. The indexer will have to work harder to scrub the data before it writes it to disk, if a Heavy Forwarder is not the intermediary.

Routing events to separate indexes

Now that we have our Heavy Forwarder, we can start collecting data. In the first case, let's use a shared firewall file, to which multiple devices write their logs.

On Linux, it's pretty easy to add the shared mount to our Heavy Forwarder, and then we can just configure our Heavy Forwarder to ingest that file. That...