Before we take a deep dive into various aspects of the Metasploit Framework, let's first lay a solid foundation of some of the absolute basics. In this chapter, we'll conceptually understand what penetration testing is all about and where the Metasploit Framework fits in exactly. We'll also browse through some of the additional tools that enhance the Metasploit Framework's capabilities.

In this chapter, we will cover the following topics:

• The importance of penetration testing
• Understanding the difference between vulnerability assessments and penetration testing
• The need for a penetration testing framework
• Introduction to Metasploit
• Introduction to new features in Metasploit 5.0
• When to use Metasploit
• Making Metasploit effective and powerful using supplementary tools

The following software is required:

• Kali Linux
• The Metasploit Framework
• Nessus
• NMAP
• w3af
• Armitage

For over a decade or so, the use of technology has been rising exponentially. Almost all businesses are partially or completely dependent on the use of technology. From Bitcoins to the cloud to the Internet of Things (IoT), new technologies are popping up each day. While these technologies completely change the way we do things, they also bring along threats with them. Attackers discover new and innovative ways to manipulate these technologies for fun and profit! This is a matter of concern for thousands of organizations and businesses around the world.

Organizations worldwide are deeply concerned about keeping their data safe. Protecting data is certainly important. However, testing whether adequate protection mechanisms have been put to work is also equally important. Protection mechanisms can fail, hence, testing them before someone exploits them for real is a challenging task. Having said this, vulnerability assessments and penetration...

Vulnerability assessments and penetration testing are two of the most common phrases that are often used interchangeably. However, it is important to understand the difference between the two. To understand the exact difference, let's consider a real-world scenario.

A thief intends to rob a house. To proceed with his robbery plan, he decides to recon his robbery target. He visits the house (that he intends to rob) casually and tries to gauge what security measures are in place. He notices that there is a window at the back of the house that is often open and so it's easy to break in. In our terms, the thief just performed a vulnerability assessment. Now, after a few days, the thief actually goes to the house again and enters through the back window that he had discovered earlier during his recon phase. In this case, the thief performed an actual penetration into his target house with...

Penetration testing is not just about running a set of a few automated tools against your target. It's a complete process that involves multiple stages and each stage is equally important for the success of the project. Now, for performing all the tasks throughout every stage of penetration testing, we would need to use various tools and might need to perform some tasks manually. Then, at the end, we would need to combine the results from all the different tools together to produce a single meaningful report. This is certainly a daunting task. It would be really easy and timesaving if one single tool could help us perform all the required tasks for penetration testing. This exact need is satisfied by a framework such as Metasploit.

Now let's move on to learning more about the Metasploit Framework.

The birth of Metasploit dates back to 16 years ago, when H. D. Moore, in 2003, wrote a portable network tool using Perl. By 2007, it was rewritten in Ruby. The Metasploit project received a major commercial boost when Rapid7 acquired the project in 2009. Metasploit is essentially a robust and versatile penetration testing framework. It can literally perform all the tasks that are involved in a penetration testing life cycle. With the use of Metasploit, you don't really need to reinvent the wheel! You just need to focus on the core objectives, the supporting actions will all be performed through various components and modules of the framework. Also, since it's a complete framework and not just an application, it can be customized and extended as per our requirements.

Metasploit is, no doubt, a very powerful tool for penetration testing. However, it's certainly not a magic wand that can help you hack into any given target system. It's...

Ever since the Metasploit Framework was born 16 years ago, it has been through significant changes and improvements. In early 2019, Metasploit 5.0 was released, which is considered its first major release since 2011. While the Metasploit is commercially supported and developed by Rapid7, it also has rich community support, which enables its growth.

The latest Metasploit 5.0 version brings in a lot more features and improvements:

• Database and automation API's: The latest Metasploit 5.0 now allow users to run the database as a RESTful service. It also introduces the new JSON-RPC API, which would be of significant help to users who wish to integrate Metasploit with other tools. The API interface can be extremely handy in several automation and orchestration scenarios. It thus makes the framework even more agile and powerful.
• Evasion modules and libraries: In 2018, a new evasion module was introduced that allowed users...

There are literally tons of tools available for performing various tasks related to penetration testing. However, most of the tools serve only one unique purpose. Unlike these tools, Metasploit can perform multiple tasks throughout the penetration testing life cycle. Before we check the exact use of Metasploit in penetration testing, let's have a brief overview of the various phases of penetration testing.

The following diagram shows the typical phases of the penetration testing life cycle:

Figure 1.1 – Phases of the penetration testing life cycle

Now let's move on to understanding the phases in detail:

• Information gathering: Though the information gathering phase may look very trivial, it is one of the most important phases for the success of a penetration testing project. The more you know about your target, the higher the chances are that you will find the right vulnerabilities and exploits to work...

So far, we have seen that Metasploit is a really powerful framework for penetration testing. However, it can be made even more useful if integrated with some other tools. This section covers a few tools that complement Metasploit's capability to perform more precise penetration on the target system. We'll start with the Nessus tool.

Nessus

Nessus is a product from Tenable Network Security and is one of the most popular vulnerability assessment tools. It belongs to the vulnerability scanner category. It is quite easy to use, and it quickly identifies infrastructure-level vulnerabilities in the target system. Once Nessus tells us what vulnerabilities exist on the target system, we can then feed those vulnerabilities to Metasploit to see whether they can be exploited for real.

Its official website is https://www.tenable.com/.

The following screenshot shows the Nessus homepage:

...

We started this chapter with understanding the relevance of penetration testing and then glanced at the practical difference between vulnerability assessment and penetration testing. We then tried to understand the exact need of a penetration testing framework and got introduced to the Metasploit Framework. We also covered the new features introduced as part of latest Metasploit 5.x Framework.

We also got an overview on when to use the Metasploit Framework in the penetration testing life cycle along with some other useful tools like Nessus, NMAP, and so on.

Now that we have got a high-level overview of what Metasploit is all about and the new features in the latest Metasploit 5.0 version, its applicability in penetration testing, and supporting tools, we'll browse through the installation and environment setup for Metasploit in the next chapter.

You can try the following exercises:

• Visit Metasploit's official website and try to learn about the differences in various editions of Metasploit.
• Try to explore more on how Nessus and NMAP can help us during a penetration test.
• Install Nessus and w3af on your Kali Linux system.

More information on the Metasploit Framework along with various versions can be found at https://metasploit.help.rapid7.com/docs.