In this chapter, we begin our journey by building a solid foundation. Having a good understanding of the basics of penetration testing will help you conduct a successful penetration test, as opposed to haphazardly scanning networks and performing tests blindly. We will define penetration testing and how it differs from other security assessments. Before the actual penetration test occurs, there are a few things that need to be done in order to ensure that the correct authorization is in place and the correct scope is defined. Every successful penetration testing student requires a lab environmentâit can be daunting to build one, but don't despair. We will look at what options exist for a lab environment.
As you progress through the chapter, you will learn the following:
- Vulnerability assessment: ThisÂ is the process of identifying vulnerabilities and risks in systems. In a vulnerability assessment, the vulnerability is not exploited. It merely highlights the risks so that the business can identify the risks and plan for remediation.
- Penetration testing: This is the authorized process of finding and using vulnerabilities to perform an intrusion into a network, application, or host in a predefined time frame. Penetration testing can be conducted by an internal team or an external third party. Penetration testing goes one step further as opposed to a vulnerability assessment, in that a penetration test exploits the vulnerability to ensure it is not a false positive.Â Penetration testing does not involve anything that is unauthorized or uncoordinated. During a penetration test, some tests might affect business applications and cause downtime. For this reason, awareness at the management and staff levels is often required.
- Red team assessment: This is similar to a penetration test, but it's more targeted. As a penetration test's main aim is to discover multiple vulnerabilities and exploit them, the goal of a red team assessment is to test an organization's response capabilities and act on vulnerabilities that will meet their goals. In a red team assessment, the team will attempt to access information in any way possible and remain as quiet as possible. Stealth is key in a red team assessment. In a red team assessment, the duration of the assessment is much longer than a penetration test.
As you start your penetration testing journey, it's important to understand what penetration testing is. To illustrate what penetration testing is, let's consider a scenario.
You currently own an organization that holds customer data. Within your organization, you have SQL databases, public-facing websites, internet-facing servers, and a sizeable number of users. Your organization is a prime target for a number of attacks, such as SQL injections, social engineering against users, and weak passwords. Should your organization be compromised, there is a risk of customer data being exposed, and more.
In order to reduce your exposure to risks, you need to identify the holes in your current security posture. Penetration testing helps you to identify these holes in a controlled manner before an attacker does. Penetration testing uses real-world attacks that attackers would leverage; the aim is to obtain accurate information as to how deep an attacker could go within your network and how much information the attacker could obtain. The results of a penetration test give organizations an open view of the vulnerabilities and allow them to patch these before an adversary can act on them.
Think of penetration testing as looking through the eyes of an enemy.
Penetration testing is often referred to as ethical hacking, white hat hacking, pentest, or pentesting.
As the security maturity of organizations differs, so will the scope of your penetration tests. Some organizations might have really good security mechanisms in place, while others might not. As businesses have policies, business continuity plans, risk assessments, and disaster recovery as integral parts of their overall security, penetration testing needs to be included.
Now that you understand what penetration testing is, you may be wondering what the flow of a penetration test is. Penetration testing has a number of stages, and each stage forms an important part of the overall penetration test.
There are various standards that relate to penetration testing. This book does not follow any one of them specifically.Â There are other known standards, such as the following:
- NIST SP800-115 standard âÂ https://csrc.nist.gov/publications/detail/sp/800-115/final
- Open Source Security Testing Methodology ManualÂ (OSSTMM) âÂ http://www.isecom.org/research
The following stages follow theÂ Penetration Testing Execution StandardÂ (PTES), which I found to be a great starting point. The full standard can be found atÂ http://www.pentest-standard.org/.
This is the most important phase in every penetration test. In this phase, you start defining the blueprint for the penetration test and align this blueprint to the business goals of the client. The aim is to ensure that everyone involved is on the same page and expectations are set well in advance.
During this phase, as a penetration tester, you need to take time to understand your client's requirements and goals. For example, why is the client performing a penetration test? Was the client compromised? Is the client performing the penetration test purely to meet a compliance requirement, or does the client intend to perform remediation on the findings? Talking to the client and understanding their business goals will help you plan and scope your penetration test so that any sticky situation can be avoided.
The pre-engagement phase consists of a few additional components that you need to consider.
This component defines what will be tested. Here, the key is in finding a balance between time, cost, and the goals of the business. It's important to note that everything agreed upon during the scope must be clearly documented and all legal implications must be considered.Â
During this component, you will ask questions such as the following:
- What is the number of IP address ranges or systems that will be tested?
- Does the penetration test cover physical security, wireless networks, application servers, social engineering, and so on?
- What is off-limits for the penetration test? The business might have mission-critical systems that could lead to loss of revenue if these are affected by the penetration test.
- Will the penetration test be onsite or offsite?
- Are there any third-party servers that are in the scope of the penetration test?Â
- Are you performing a white-box, grey-box, or black-box penetration test?
The questions listed do not cover everything, and the questions will vary per client. To get a more comprehensive list of the type of questions you should consider, you can refer to the PTES Standard atÂ http://www.pentest-standard.org/index.php/Pre-engagement.
White-box testing gives you complete open access to systems, code, network diagrams, and so on. It provides more comprehensive results that are not available to average attackers.Grey-box testing gives you some sort of information about the internal systems; the aim is to obtain information from the viewpoint of an attacker who has already breached the system.Black-box testing does not provide you with any information or access to the network. This type of test is more practical, as you simulate an external attacker.
While you work on scoping your penetration test, be very careful ofÂ scope creep.Â Scope creepÂ is any additional work that is not agreed upon during the initial scope. ItÂ introduces risks to your penetration test, which can lead to loss of revenue for you, an unsatisfied client, and even legal implications. Scope creep is a trap that you can easily fall into.
Keep in mind the cost of a penetration test when in the scoping phase. Penetration test prices vary depending on what needs to be tested. For example, testing a complex web application will require a lot more time and effort, therefore the cost will be a lot more when compared to a simple network penetration test.Â The regularity with which you conduct the penetration test is another factor that affects the cost.
Timelines can be set by the client as to when you are allowed to perform the penetration test. Some clients might have business-critical servers that are patched during a specific time window, and these servers might be off-limits during that time.
Ensure that the start and end dates are defined. This allows the penetration test to have a defined end date.
Today, many businesses are utilizing cloud services. There is a high probability that you will encounter cloud servers within your penetration scope. It's important to keep in mind who owns the server. In the case of a cloud environment, the server is not owned by the business that the penetration test is being conducted for, but rather the cloud provider.
Big players in the cloud space, such as Microsoft, Amazon, and Google, all have penetration testing rules-of-engagement documents. These documents detail what you are allowed to do and what you are not allowed to do.
Microsoft defines its rules of engagement here:Â https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement.https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement Amazon defines its rules of engagement here:Â https://aws.amazon.com/security/penetration-testing/.https://aws.amazon.com/security/penetration-testing/ Google defines its rules of engagement here:Â https://cloud.google.com/security/overview/.
Make sure that you obtain the correct approvals from the cloud provider if you have any cloud services within your penetration scope; failure to do so might lead to legal consequences.
Discussions around payment terms are crucial, as it's common for large organizations to delay payments. You need to define your payment terms upfront. Clear dates should be defined as to when payments should be made.Â
Don't forget to define the costs; for example, you will perform a penetration test on 10 IP addresses at a cost of $500 per IP address.Â
As you perform penetration testing, you will uncover multitudes of information that are valuable to real-world attackers, and you will also be performing activities that are illegal. The only thing that separates a penetration tester from a malicious hacker is permission.
Obtaining the relevant permission forms your "get out of jail free card". The permission that is provided by the business details any constraints and authorizes you to perform activities defined in your scoping agreement.Â
It's a formal approval from the business to begin the penetration test.
Once you have completed the pre-engagement phase, you need to gather as much information as you can before you begin your attack. In the intelligence-gathering phase, also referred to as information gathering, you start looking at how much information you can obtain about your target. You will gather information from publicly accessible resources. This is known as Open Source Intelligence (OSINT). You will start leveraging tools that can assist you, such as Maltego and Shodan.
The importance of intelligence gathering is that you are able to detect entry points into the target organization. Businesses and employees do not take into account how much of their data they can expose on the internet, so this data becomes a wealth of information for a determined attacker.
In Chapter 3,Â Performing Information Gathering, we will cover information gathering in more detail.
Once you have gathered information inÂ the intelligence-gathering phase, you start working on threat modeling. In threat modelling, you begin to create a structure of threats and how they relate to your target's environment. For example, you will identify systems that hold valuable information, then you will identify the threats that pertain to the systems and what vulnerabilities exist in the system that can allow the attacker to act on the threat.
- Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege (STRIDE)
- Process for Attack Simulation and Threat Analysis (PASTA)
- Visual Agile and Simple Threat Modelling (VAST)
- Microsoft Threat Modelling Tool:Â https://aka.ms/tmt
- OWASP Threat Dragon:Â https://www.owasp.org/index.php/OWASP_Threat_Dragon
Once you have defined the threats that could lead to compromise, it's time to discover what vulnerabilities exist for those threats. In the vulnerability analysis phase, you start to discover vulnerabilities in systems and how you can act upon those by using exploits.Â
Here, you will perform either active or passive analysis. Keep in mind that any failed exploits can lead to detection.
Active vulnerability analysis can consist of the following:
- Network scanners
- Web application scanners
- Automated scanners
Passive vulnerability analysis can consist of the following:
- Monitoring traffic
There are many vulnerability scanners that exist today. For example, the more commonly used one is Nessus, but there are many others, such as OpenVAS, Nikto, and QualysGuard.
In the exploitation phase, you start focusing on obtaining access to systems and evading any security blockers that exist. By performing a vulnerability analysis in the exploitation phase, you can create a precise plan that you can execute.Â
In this phase, you will begin to work with many tools. Some exploits can be done easily, while others can be complex.Â
The post-exploitation phase really showcases your skills as a penetration tester. When malicious hackers breach a system, they start to trawl the environment looking for high-value targets. They also start creating backdoors so that they can easily revisit the compromised system.
As a penetration tester, you would perform tasks as if you were an attacker. Once you have breached a system, it's time to look for high-value targets and valuable information, attempt to access escalated privileges, move laterally, and look at how you can pivot.
In the final phase of penetration testing, findings need to be provided to the business in a meaningful way. Here, you would define everything from how you entered their environment to what you found. It's important to provide the business with recommendations on how to fix the gaps that you have exposed in your penetration test.
Your report should have an executive summary and a technical report. Each section needs to be tailored to the audience that you are presenting it to. For example, you would not say that you used the MS17-010 EternalBlue exploit to compromise a system in the executive summary, but you would say this in the technical report.
The executive summary will define the goals of the penetration test and provide an overview of the findings at a very high level. As the audience of the executive summary is usually the business decision-makers, you need to communicate on their level. In order to do that, the executive summary may contain the following sections:
- Background:Â In the background section, you need to explain the purpose of the penetration test.Â
- Overall posture:Â Here, you will define how effective the penetration test was in relation to the goals defined during the pre-engagement phases.
- Risk ranking:Â This defines the overall risk rating that the business resides in. For example, the business might be at an extreme, high, moderate, orÂ low risk. You have to explain this rating so that it is clear to the business why they fall into that risk rank.
- General findings:Â This section provides a brief summary of the issues that were identified during the penetration test. Charts are often found here that highlight security risk categories; for example, missing patches and operating system hardening.
- Recommendation summary: This outlines a high-level overview of what tasks should be performed to re-mediate the findings. Do not go into detail here, as details are covered in the technical report.
- Strategic roadmap: This provides the business with an actionable roadmap to remediate the findings. This roadmap must be prioritized and be in line with the business-level of potential impact. The roadmap can be broken down into parts, such as 1 to 3-month, 3 to 6-month, and 6 to 12-monthÂ plans. Within each section, there should be actions defined; for example, within the 1 to 3-monthÂ plan, the business should address missing patches that are low-impact.
The technical report will includeÂ a lot more details compared to the executive summary. In the technical report, you will define the scope, information, attack methods, and remediation steps in full. In this report, you can use technical terms that are easily understood, such asÂ remote shell, pass-the-hash, and NTLM hashes.
- Introduction:Â This part will include topics such as the scope of the penetration test, contacts, systems involved, and approach.
- Information gathering:Â Here, you will explain how much of information you were able to gather on the targets. In this section, you can dive deeper to highlight what information was obtained by passive intelligence (informationÂ publicly availableÂ on the internet, DNS records, IP address information,Â and so on), active intelligence (port scanning, footprinting,Â and so on), personnel intelligence (what information was obtained from social engineering, phishing,Â and so on), and so forth.
- Vulnerability assessment:Â In this section, you will define what types of vulnerabilities were discovered, how they were discovered, and provide evidence of the vulnerability.Â
- Exploitation/vulnerability verification:Â This section provides the detailed steps on how you acted on the vulnerabilities discovered. Details such as a timeline of the attack, targets, success/fail ratio, and level of access obtained should be included.
- Post exploitation:Â Details included here would be activities such as escalation paths, data extraction, information value, how effective the countermeasures were (if any), persistence, and pivot points.
- Risk/exposure:Â The results from the preceding sections are combined and tied to a risk and exposure rating. This section would contain information such as estimated loss per incident, the skill required to perform a certain attack, countermeasure strength, and risk ranking (critical, high, medium, low).
- Conclusion:Â The conclusion should always end on a positive note. Here, you will highlight any guidance for increasing the business' security posture with a final overview of the penetration test.
Now that we have built our foundation on what penetration testing is, its phases, and how it differs from vulnerability assessments and red team assessments, it's time to dive into lab environments.
There are three options that we have for building a penetration lab. These are as follows:
- Using a cloud provider:Â Cloud providers such as Microsoft Azure, Amazon Web Services, and Google Cloud give you the flexibility and scalability of deploying systems at a fraction of the cost compared to purchasing dedicated hardware. The only catch with using a cloud provider is that you would probably require permission to perform penetration tests on your deployed services.
- Using a high-powered laptop or desktop with virtualization software:Â As high-powered laptops and desktops are relatively cheap, this would be the option that many prefer. By using virtualization software such as Microsoft Hyper-V, VMware, and Virtualbox, you can deploy a fully isolated network on your host computer.
When using a hypervisor for penetration testing, there is a limitation with Hyper-V. Currently, Hyper-V does not allow you to connect a USB wireless card directly to the VM, as opposed to VMware, shown in the following screenshot, and Virtualbox. This introduces problems when you try to leverage monitor mode for wireless penetration testing. VMware and Virtualbox allow you to connect a USB wireless card directly to the virtual machine. The following screenshotÂ depicts connecting a wireless network card directly to the virtual machineÂ (Figure 1):
Figure 1: Connecting a wireless card to the virtual machine.
Let's start by looking at building a lab environment using virtualization tools such as VMware, Hyper-V, and VirtualBox.
Leveraging a hypervisor enables you to build your lab environment with minimal hardware costs. Any decent laptop or desktop these days is able to run hypervisor software. When you use your hypervisor of choice, make sure that you configure the virtual networks appropriately. For example, if you require your VMs to be isolated, then you would use host only. If you require your virtual machine to have internet access, you could use network address translation or bridged networking.
The difference between network address translation and bridged networking is that, with bridged networking, your virtual machine will obtain its own IP address, whereas with network address translation, your virtual machine will leverage your hosts, IP address to communicate externally.
Note that the options might differ between the different pieces of hypervisor software, but the concepts are the same.
Hyper-V is a virtualization product by Microsoft that you can use to create VMs.Â Microsoft Hyper-V is available on Windows 10. It can be enabled on Enterprise, Education, and Pro versions of Windows 10.Â
Hyper-V can be installed a number of ways.
Using Powershell, you can install the Hyper-V role using the following code:
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All
Also, by using the following DISM and CMD (running as administrator) code:
DISM /Online /Enable-Feature /All /FeatureName:Microsoft-Hyper-V
Hyper-V can also be installed using programs and features within Windows. To do so, the following steps should be performed in Windows 10:
- Â Press the Windows key + R to open the
Rundialog box. Type in
Programs and Featuresas shown inÂ Figure 2:
Figure 2: Opening the Programs and Features.
- Click on
TurnÂ Windows features on or offÂas shown inÂ Figure 3:
Figure 3: Turn windows features on or off.
- Select the
Hyper-Vroles as shown inÂ Figure 4:
Figure 4: Selecting Hyper-V roles.
- Click on
Your computer will require a reboot to install the Hyper-V roles.
Hyper-V currently does not have the ability to connect a USB device directly to the virtual machine. This introduces problems with wireless cards that will be used for wireless penetration testing, as you are unable to switch to monitor mode.
More information about Microsoft Hyper-V can be found atÂ https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/about/.
VMware Workstation Player (https://www.vmware.com/products/workstation-player.html) is available for free for Microsoft Windows and Linux operating systems. There is a paid version called VMware Workstation Pro (https://www.vmware.com/products/workstation-pro.html), which offers some additional features over the free version.Â VMware Fusion (https://www.vmware.com/products/fusion.html), which is also a paid version, is available for macOS.Â
Virtualbox is an open source hypervisor that is free to use. It offers support for Microsoft Windows, Linux, and macOS. Virtualbox has a number of extensions that are available for use, which includes support for USB3, PXE boot, disk encryption, and more.
Virtualbox can be downloaded atÂ https://www.virtualbox.org/wiki/Downloads.
When we talk about target machines, these are VMs that will be used to test various tools and concepts in this book.
For Microsoft Windows, we will leverage the evaluation center to download Windows 10 Enterprise and Server 2012 R2.
The Microsoft evaluation center can be accessed atÂ https://www.microsoft.com/en-us/evalcenter/.
The direct link for Windows Server 2012R2 is https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2012-r2, and for Windows 10 Enterprise, the direct link isÂ https://www.microsoft.com/en-us/evalcenter/evaluate-windows-10-enterprise.
For both operating systems, you will leverage the
.iso file and install them using your hypervisor of choice.
Metasploitable is an intentionally vulnerable machine that you can use to test Metasploit exploits to obtain shell permissions. Metasploitable differs from other vulnerable machines, since it focuses more on the operating system and network layer.
Metasploitable currently has three versions to date; these are aptly named Metasploitable, Metasploitable 2, and Metasploitable 3.
There are significant changes in each release over and above how you would set them up.
Metasploitable (version 1) is a VM-based customized Ubuntu image. Within this image, there is a number of vulnerable and poorly configured software installed. For example, you might have Tomcat with weak credentials, easily exploitable using Metasploit.
Metasploitable (version 1) is available on Vulnhub for download atÂ https://www.vulnhub.com/entry/metasploitable-1,28/.
Metasploitable 2, which was more robust, had a lot more vulnerabilities introduced. It included more than 30 exposed ports that would show up in a Nmap scan. It also included vulnerable web applications, such as Damn Vulnerable Web App (DMVA) and Mutillidae. This allowed people to test their web application penetration testing skills.
Metasploitable (version 2) is available for download atÂ https://information.rapid7.com/download-metasploitable-2017.html.
Metasploitable 3 upped the game. Versions 1 and 2 were Linux-based, but version 3 is Windows-based. Metasploitable 3 makes use of automation and provisioning. The build process is simple and robust, all of its scripts are open source, and it leverages tools such as vagrant and packer. At the time this book was written, Metasploitable 3 supports both VMware and Virtualbox.Â
Metasploitable 3 is available for download fromÂ https://github.com/rapid7/metasploitable3.
- Packer (available for download atÂ https://www.packer.io/intro/getting-started/install.html)
- Vagrant (available for download atÂ https://www.vagrantup.com/docs/installation/)
- The Vagrant reload plugin (available for download atÂ https://github.com/aidanns/vagrant-reload#installation)
- Virtualbox or VMwareÂ
- Metasploitable 3 (available for download at https://github.com/rapid7/metasploitable3)
The build steps for Metasploitable 3 are relatively simple and can be found on the GitHub repository maintained by Rapid7 (https://github.com/rapid7/metasploitable3). There are resources available on the internet that host pre-built Metasploitable 3 VMs.
In Chapter 5, Diving into the Metasploit Framework, we will perform various tasks using the Metasploitable labs discussed here.
In this chapter, we began to build a solid foundation as you learned about penetration testing and how it differs from vulnerability and red team assessments. We defined the importance of leveraging a methodology or standard for penetration testing, such as the PTES standard and the various phases within it. Within each phase, we discussed what is involved and highlighted some important facets that should not be overlooked, such as the "get out of jail free card". Lastly, we looked at a lab environment, the various hypervisors that currently exist, and how to build a Metasploitable lab environment for future use.
In the next chapter (Chapter 2, Getting Started with Kali Linux), we will dive into Kali Linux to explore the different install options for Kali Linux and some initial configuration that is required. We will explore some of the essential tools within Kali Linux and what their main uses are.