Network Anomaly Detection with AI

The current level of interconnection that can be established between different devices (for example, think of the Internet of Things (IoT)) has reached such a complexity that it seriously questions the effectiveness of traditional concepts such as perimeter security. As a matter of fact, cyberspace's attack surface grows exponentially, and it is therefore essential to resort to automated tools for the effective detection of network anomalies associated with unprecedented cybersecurity threats.

This chapter will cover the following topics:

• Network anomaly detection techniques
• How to classify network attacks
• Detecting botnet topology

• Different machine learning (ML) algorithms for botnet detection

In this chapter, we will focus on anomaly detection related to network security, postponing the discussion of the aspects of fraud detection...

Network anomaly detection techniques

The techniques we have seen so far can also be adopted to manage anomaly detection and related attempts to gain unauthorized access to the corporate network. To fully understand the potential of anomaly detection techniques, we will trace its evolution in the cybersecurity area, illustrating the basic principles that characterize it.

In fact, anomaly detection has always been a research area of cybersecurity, particularly in the field of network security protection. However, anomaly detection is not limited to identifying and preventing network attacks, but can also be adopted in other areas, such as fraud detection and in the identification of possible compromises of user profiles.

Anomaly detection rationales

...

How to classify network attacks

We have seen that it is possible to use all different types of algorithms (such as supervised, unsupervised, and reinforcement learning), even in the implementation of network anomaly detection systems.

But how can we effectively train these algorithms in order to identify the anomalous traffic?

It will be necessary to first identify a training dataset that is representative of the traffic considered normal within a given organization.

To this end, we will have to adequately choose the representative features of our model.

The choice of features is of particular importance, as they provide a contextual value to the analyzed data, and consequently determine the reliability and accuracy of our detection system.

In fact, choosing features that are not characterized by high correlation with possible anomalous behaviors translates into high error rates...

Detecting botnet topology

One of the most common pitfalls in network anomaly detection has to do with the detection of botnets within the corporate network. Given the danger of such hidden networks, the detection of botnets is particularly relevant, not only for preventing the exhaustion of the organization's computational and network resources by external attackers, but also for preventing the dissemination of sensitive information (data leakage) outward.

However, identifying the presence of a botnet in time is often an operation that is anything but simple. This is why it is important to understand the very nature of botnets.

What is a botnet?

The term botnet comes from the juxtaposition of the words bot and net. In...

Different ML algorithms for botnet detection

From what we have described so far, it is clear that it is not advisable to exclusively rely on automated tools for network anomaly detection, but it may be more productive to adopt AI algorithms that are able to dynamically learn how to recognize the presence of any anomalies within the network traffic, thus allowing the analyst to perform an in-depth analysis of only really suspicious cases. Now, we will demonstrate the use of different ML algorithms for network anomaly detection, which can also be used to identify a botnet.

The selected features in our example consist of the values of network latency and network throughput. In our threat model, anomalous values ​​associated with these features can be considered as representative of the presence of a botnet.

For each example, the accuracy of the algorithm is calculated...

Summary

In an increasingly interconnected world, and with the progressive spread of the IoT, it becomes essential to effectively analyze network traffic in search of anomalies that can represent reliable indications of possible compromises (such as the presence of botnets).

On the other hand, the exclusive use of automated systems in performing network anomaly detection tasks exposes us to the risk of having to manage an increasing number of misleading signals (false positives).

It is, therefore, more appropriate to integrate the automated anomaly detection activities with analysis carried out by human operators, exploiting AI algorithms as filters, in order to only select the anomalies that are really worthy of in-depth attention from the analysts.

In the next chapter we will deal with AI solutions for securing user authentication.

...