Chapter 15: Network Intrusion (Detection and Prevention) Systems

In this chapter, we will explore the Intrusion Detection System (IDS) and Intrusion Prevention System (IPS) concepts to understand their functionality. This will help us implement a good network perimeter defense using them. We will explore how OPNsense employs Suricata and combines it with Netmap to implement an outstanding IDS and IPS open source solution. By the end of this chapter, you will know how to use an IDS/IPS solution to monitor and block traffic using OPNsense.

In this chapter, we will cover the following topics:

• IDS and IPS definition
• Suricata and Netmap
• Rulesets
• Configuration
• SSL fingerprint
• Troubleshooting

Technical requirements

Good TCP/IP networking knowledge will be enough for you to understand the concepts in this chapter. To follow the configuration steps, you will need a working version of OPNsense with a host connected to it, along with an active internet connection.

IDS and IPS definition

The rise of different types of attacks on the internet has pushed firewall solutions to increase their defense mechanisms. A layer 4-only approach became inefficient against more sophisticated attacks, such as techniques that are used to exploit a known vulnerability, requiring a new approach to detect and block the latest threats. Let's look at a practical example.

Suppose that a layer 4-only firewall allows LAN to internet-only connections to well-known internet protocols such as DNS and HTTP, and connections that are used for email communication, such as POP3, SMTP (submission), and IMAP. So, to bypass the firewall, the attacker could install malware that uses the same ports as the HTTP protocol, such as 80 and 443, so that the malware can transmit data using an allowed port without any problem. Now, suppose that this kind of technique became a trend, and all attackers started bypassing firewalls in this way! Cybersecurity professionals would get...

Suricata and Netmap

The OPNsense IDPS implementation is based on the Suricata project, a truly open source I(DP)S that's supported by the Open Information Security Foundation (OISF). Suricata is an excellent open source NIDS solution with superb support for signatures from companies such as Proofpoint, for example. On OPNsense, Suricata has Netmap support, which means fewer CPU resource requirements to detect threats, which results in good performance. The Netmap framework is driver-dependent, and it is essential to check whether the network device that's being used supports it before activating a feature that uses Netmap. In this chapter, we will do that while enabling IPS mode.

Note

We discussed how to implement Netmap in OPNsense in Chapter 2, Installing OPNsense. There, you can find the Netmap devices that are supported on FreeBSD.

The OPNsense project steps closer to the OISF, so you can always expect a better Suricata implementation on it. The Suricata and...

Rulesets

The I(DP)S rulesets are a group of rules that you can enable to detect certain types of traffic – for example, a signature that's been designed to prevent attacks on web servers. In OPNsense, it is possible to enable different rulesets simultaneously. By default, the available rulesets are as follows:

• Abuse.ch: These are rulesets that are provided by the Abuse.ch project. They focus on blacklists based on an IP address's reputation.
• Proofpoint's Emerging Threat Open (ET Open): This is the community version of the Proofpoint ruleset. It's more limited than the ET Pro version.
• OPNsense Application Detection: This is OPNsense's project ruleset. It contains rules for controlling web applications such as YouTube, Netflix, Dropbox, and others.

Whatever ruleset you decide to use, you must download and install these rulesets before enabling them.

Some additional rulesets that are available as plugins are as follows:

...

Configuration

To configure an I(DP)S, we need to go to Services | Intrusion Detection | Administration and go to the Settings tab. This can be seen in the following screenshot:

Figure 15.3 – IDS configuration page

Here, you will see the following options:

• Enabled: Check this option to enable the IDS service.
• IPS mode: Enabling this option will turn the IDS into an IPS service, blocking traffic instead of only alerting you about it. Disable the hardware offloading options for network interfaces by going to the Interfaces | Settings page to avoid network issues while using IPS mode. Check the Hardware CRC, Hardware TSO, and Hardware LRO options, and then ensure that Disable VLAN Hardware Filtering is also disabled.

  Important Note

  The Suricata daemon in IPS mode uses Netmap in OPNsense. It doesn't work well with hardware offloading, so keep it disabled while using IPS mode.

• Promiscuous mode: This mode allows all the traffic to be...

SSL fingerprint

The Suricata project that's used by OPNsense, known as a192.168.0.1s, has an I(DP)S engine that isn't capable of decrypting SSL/TLS traffic without external tools. Still, you can create your own rules based on the SSL/TLS certificates' SHA fingerprints. SHA is an algorithm that checks data integrity, and in OPNsense, we can extract it from a website certificate to match traffic and create custom rules for alerting or even blocking traffic.

Note

This isn't a stable feature. Test it before implementing it in a production environment. For more accurate SSL/TLS filtering in a production environment, it is advisable to use the Zenarmor plugin instead.

To start, go to Services | Intrusion Detection | Administration. Then, go to the User defined tab and click + to create a new custom rule. The following screenshot shows the Rule details page:

Figure 15.14 – The Rule details page

On the Rule details page, you can...

Troubleshooting

In this section, we will explore some of the common issues you may face while using an I(DP)S in OPNsense and also how to solve each:

• Poor performance while using IPS: First things first – plan your IPS deployment with care! IPS mode is a resource-consuming feature, and you will need suitable hardware with a high-clock CPU, multicores, and good network cards to get the IPS working well in a high-traffic network. The recommended pattern matcher is Hyperscan, which works well with supported CPUs (SSE3-capable). A long list of enabled rulesets will demand a lot of RAM if you wish to inspect traffic without issues. A common symptom of this issue is high CPU usage and low bandwidth, especially from WAN, while using IPS mode. If your hardware isn't working reasonably with IPS mode on, it might be good to keep it only in IDS mode, which will consume fewer resources.
• A lot of false positives: A good deployment plan will avoid this kind of situation....

Summary

In this chapter, you learned about IDSs and IPSs and how to configure OPNsense as an IDS to gather traffic alerts for traffic based on rulesets. You also learned about IPS mode, as well as how to enable it and block traffic on the local network. In the Troubleshooting section, we explored common issues that you may face while using an I(DP)S in OPNsense and learned how to solve each.

In the next chapter, we'll take traffic inspection to the next level by using the next-generation capabilities that are available via Sunny Valley's Zenarmor plugin.