Download code from GitHub
Web applications are everywhere. They are part of the fabric of society and we depend on them in many aspects of our lives. Nowadays, they are easy to develop, quick to deploy, and accessible by anyone with an internet connection.
The technology designed to help develop and deploy web applications has also boomed. New frameworks that enhance functionality and usability are released daily. Companies have shifted power to the developer, allowing them to be more agile and produce web applications quickly.
The following figure gives a taste of the more popular development environments and frameworks that have taken the application development world by storm. Node.js has brought the browser client scripting language JavaScript to the server-side, complete with a massive library of modules to aid in fast application development. JavaScript, a once seldom-used scripting language for the browser, is supercharged on the client-side with React and Angular, and is even available for cross-platform development with the likes of Electron and Chromium:
Figure 1.1: The world has changed since Netscape ruled online and this graphic shows but a taste of the technologies that dominate the web today
GitHub has become the one-stop shop for open-source libraries, applications, and anything a developer may want to share with the world. Anyone can upload anything they want and others can collaborate by pushing code changes or saving a dying codebase, by forking it and continuing development locally. GitHub is not alone, of course, as there are similar repositories for Node.js, Python, and PHP modules.
The developer's focus is always on getting the product shipped, whether it's a simple feature implementation in an internal web application used by the marketing department, or the latest and greatest web banking interface. The infrastructure required to support these applications has also evolved and developers struggle to integrate security into their workflow. It's not always ignorance that hurts secure application development, however. More often than not, time constraints and deadlines are to blame.
The goal of this book is to showcase how attackers view web applications and how they take advantage of weaknesses in the application code and infrastructure. We will consider all the common mistakes made during the development process that are used to gain meaningful access. We will look at practical attacks and making the most of common application vulnerabilities.
Some assumptions about your knowledge level are made. To get the most value out of reading this book, a basic knowledge of application security should be there. Readers do not have to be experts in the field of penetration testing or application security, but they should have an idea about what cross-site scripting (XSS) or SQL injection (SQLi) attacks are. We will not devote a chapter to the standard "Hello World" example for XSS, but we will show the impact of exploiting such a vulnerability. The reader should also be familiar with the Linux command prompt and common console tools, such as curl, git, and wget. Some familiarity with programming will certainly help, but it is not a hard requirement.
In this chapter, we will cover the following topics:
The typical rules of engagement when conducting a test
The tester's toolkit
Attack proxies
How the cloud can help with engagements
Before moving forward with the fun stuff, it is important to always remember the rules of engagement (ROE) when conducting an attack. The ROE are typically written out in the pre-engagement statement of work (SoW) and all testers must adhere to them. They outline expectations of the tester and set some limits to what can be done during the engagement.
While the goal of a typical penetration test is to simulate an actual attack and find weaknesses in the infrastructure or application, there are many limitations, and with good reason. We cannot go in guns blazing and cause more damage than an actual adversary. The target (client), be they a third party or an internal group, should feel comfortable letting professional hackers hammer away at their applications.
Good communication is key to a successful engagement. Kickoff and close-out meetings are extremely valuable to both parties involved. The client should be well aware of who is performing the exercise, and how they can reach them, or a backup, in case of an emergency.
The kickoff meeting is a chance to go over all aspects of the test, including reviewing the project scope, the criticality of the systems, any credentials that were provided, and contact information. With any luck, all of this information was included in the scoping document. This document's purpose is to clearly outline what parts of the infrastructure or applications are to be tested during the engagement. The scope can be a combination of IP ranges, applications, specific domains, or URLs. This document is usually written with the input of the client, well in advance of the test start date. Things can change, however, and the kickoff is a good time to go over everything one last time.
Useful questions to clarify during the kickoff meeting are as follows:
Has the scope changed since the document's last revision? Has the target list changed? Should certain parts of the application or network be avoided?
Is there a testing window to which you must adhere?
Are the target applications in production or in a development environment? Are they customer-facing or internal only?
Are the emergency contacts still valid?
If credentials were provided, are they still valid? Now is the time to check these again.
Is there an application firewall that may hinder testing?
The goal is generally to test the application and not third-party defenses. Penetration testers have deadlines, while malicious actors do not.
Note
When testing an application for vulnerabilities, it is a good idea to ask the client to whitelist out IPs in any third-party web application firewalls (WAFs). WAFs inspect traffic reaching the protected application and will drop requests that match known attack signatures or patterns. Some clients will choose to keep the WAF in an enforcing mode, as their goal may be to simulate a real-world attack. This is when you should remind the clients that firewalls can introduce delays in assessing the actual application, as the tester may have to spend extra time attempting to evade defenses. Further, since there is a time limit to most engagements, the final report may not accurately reflect the security posture of the application.
Note
No manager wants to hear that their critical application may go offline during a test, but this does occasionally happen. Some applications cannot handle the increased workload of a simple scan and will failover. Certain payloads can also break poorly-designed applications or infrastructure, and may bring productivity to a grinding halt.
Note
If, during a test, an application becomes unresponsive, it's a good idea to call the primary contact, informing them of this as soon as possible, especially if the application is a critical production system. If the client is unavailable by phone, then be sure to send an email alert at minimum.
Close-out meetings or post-mortems are also very important. A particularly successful engagement with lots of critical findings may leave the tester feeling great, but the client could be mortified, as they must explain the results to their superiors. This is the time to meet with the client and go over every finding, and explain clearly how the security breach occurred and what could be done to fix it. Keep the audience in mind and convey the concerns in a common language, without assigning blame or ridiculing any parties involved.
Engagements that involve any kind of social engineering or human interaction, such as phishing exercises, should be carefully handled. A phishing attack attempts to trick a user into following an email link to a credential stealer, or opening a malicious attachment, and some employees may be uncomfortable being used in this manner.
Before sending phishing emails, for example, testers should confirm that the client is comfortable with their employees unknowingly participating in the engagement. This should be recorded in writing, usually in the SoW. The kickoff meeting is a good place to synchronize with the client and their expectations.
Unless there is explicit written permission from the client, avoid the following:
Do not perform social engineering attacks that may be considered immoral, for example, using intelligence gathered about a target's family to entice them to click on a link
Do not exfiltrate medical records or sensitive user data
Do not capture screenshots of a user's machines
Do not replay credentials to a user's personal emails, social media, or other accounts
Note
Some web attacks, such as SQLi or XML External Entity (XXE), may lead to data leaks, in which case you should inform the client of the vulnerability as soon as possible and securely destroy anything already downloaded.
While most tests are done under non-disclosure agreements (NDAs), handling sensitive data should be avoided where possible. There is little reason to hold onto medical records or credit card information after an engagement. In fact, hoarding this data could put the client in breach of regulatory compliance and could also be illegal. This type of data does not usually provide any kind of leverage when attempting to exploit additional applications. When entering proof in the final report, extra care must be taken to ensure that the evidence is sanitized and that it contains only enough context to prove the finding.
"Data is a toxic asset. We need to start thinking about it as such, and treat it as we would any other source of toxicity. To do anything else is to risk our security and privacy."
- Bruce Schneier
The preceding quote is generally aimed at companies with questionable practices when it comes to private user data, but it applies to testers as well. We often come across sensitive data in our adventures.
A successful penetration test or application assessment will undoubtedly leave many traces of the activity behind. Log entries could show how the intrusion was possible and a shell history file can provide clues as to how the attacker moved laterally. There is a benefit in leaving breadcrumbs behind, however. The defenders, also referred to as the blue team, can analyze the activity during or post-engagement and evaluate the efficacy of their defenses. Log entries provide valuable information on how the attacker was able to bypass the system defenses and execute code, exfiltrate data, or otherwise breach the network.
There are many tools to wipe logs post-exploitation, but unless the client has explicitly permitted these actions, this practice should be avoided. There are instances where the blue team may want to test the resilience of their security information and event monitoring (SIEM) infrastructure (a centralized log collection and analysis system), so wiping logs may be in scope, but this should be explicitly allowed in the engagement documents.
That being said, there are certain artifacts that should almost always be completely removed from systems or application databases when the engagement has completed. The following artifacts can expose the client to unnecessary risk, even after they've patched the vulnerabilities:
Web shells providing access to the operating system (OS)
Malware droppers, reverse shells, and privilege escalation exploit payloads
Malware in the form of Java applets deployed via Tomcat
Modified or backdoored application or system components:
Example: overwriting the password binary with a race condition root exploit and not restoring the backup before leaving the system
Stored XSS payloads: this can be more of a nuisance to users on production systems
Not all malware introduced during the test can be removed by the tester. Cleanup requires reaching out to the client.
Note
Make a note of all malicious files, paths, and payloads used in the assessment. At the end of the engagement, attempt to remove as much as possible. If anything is left behind, inform the primary contact, providing details and stressing the importance of removing the artifacts.
Note
Tagging payloads with a unique keyword can help to identify bogus data during the cleanup effort, for example: "Please remove any database records that contain the keyword: 2017Q3TestXyZ123."
A follow-up email confirming that the client has removed any lingering malware or artifacts serves as a reminder and is always appreciated.
The penetration testing tools used vary from professional to professional. Tools and techniques evolve every day and you have to keep up. While it's nearly impossible to compile an exhaustive list of tools that will cover every scenario, there are some tried-and-true programs, techniques, and environments that will undoubtedly help any attacker to reach their goal.
Previously known as BackTrack, Kali Linux has been the Linux distribution of choice for penetration testers for many years. It is hard to argue with its value, as it incorporates almost all of the tools required to do application and network assessments. The Kali Linux team also provides regular updates, keeping not only the OS but also the attack tools current.
Kali Linux is easy to deploy just about everywhere and it comes in many formats. There are 32-bit and 64-bit variants, portable virtual machine packages, and even a version that runs on the Android OS:
Figure 1.2: A fresh instance of the Kali Linux screen
One alternative or supplement to Kali Linux is the Penetration Testing Framework (PTF) from the TrustedSec team and it is written in Python. This is a modular framework that allows you to turn the Linux environment of your choice into a penetration testing toolset. There are hundreds of PTF modules already available, and new ones can be quickly created. PTF can also be run on Kali to quickly organize existing tools in one location.
Figure 1.3: The PTF interactive console
Another well-established alternative to Kali Linux is BlackArch, a distribution based on Arch Linux that includes many of the tools bundled with other penetration testing distributions. BlackArch has many of the tools that testers are familiar with for network testing or application assessments, and it is regularly updated, much like Kali Linux. For Arch Linux fans, this is a welcome alternative to the Debian-based Kali distribution.
Figure 1.4: The main BlackArch screen
BlackArch is available in many formats on https://blackarch.org.
When testing applications, traffic manipulation and recording is invaluable. The major players in this market are also extendable, allowing the community of researchers to improve functionality with free add-ons. Well-built and supported proxies are powerful weapons in the attacker's arsenal.
Burp Suite is arguably the king when it comes to attack proxies. It allows you to intercept, change, replay, and record traffic out of the box. Burp Suite is highly extendable, with powerful community plugins that integrate with sqlmap (the de facto SQLi exploitation tool), automatically test for privilege escalation, and offer other useful modules:
Proxy: Record, intercept, and modify requests on the fly
Spider: Content discovery with powerful crawling capabilities
Decoder: Unscramble encoded data quickly
Intruder: A highly customizable brute-forcing module
Repeater: Allows the replay of any request previously recorded, with the ability to modify any part of the request itself
Scanner (pro only): A vulnerability scanner that integrates with Burp Collaborator to find obscure vulnerabilities
Collaborator: Aids in the discovery of obscure vulnerabilities, which would normally be missed by traditional scanners
There is a free version of Burp Suite, but the professional edition of the product is well worth the investment. While the free version is perfectly usable for quick tests, it does have some limitations. Notably, the Intruder module is time-throttled, making it useless for large payloads. The Scanner module is also only available in the professional version and it is worth the price. Scanner can quickly find low-hanging fruit and even automatically leverage Collaborator to find out-of-band vulnerabilities. The free version can still intercept, inspect, and replay requests, and it can also alert of any vulnerabilities it has passively detected.
Figure 1.5: The main Burp Suite Free Edition screen
OWASP's Zed Attack Proxy (ZAP) is another really great attack proxy. It is extendable and easy to use. However, it lacks some of the features of Burp Suite; for example, ZAP does not have the extensive active vulnerability scanning capabilities of Burp Suite Pro, nor does it have an automated out-of-band vulnerability discovery system comparable to Collaborator.
However, there is no time-throttling on its version of the Intruder module and all of its features are available out of the box. ZAP is open-source and it is actively worked on by hundreds of volunteers.
Figure 1.6: The main ZAP screen
When conducting assessments, it is common for an attacker to leverage command and control (C2) servers during a campaign. The purpose of most C2 servers is to issue commands to malware running inside the compromised environment.
Attackers can instruct malware to exfiltrate data, start a keylogger, execute arbitrary commands or shellcode, and much more. In later chapters, we will primarily use the cloud C2 server to exfiltrate data and to discover vulnerabilities out-of-band.
A C2 server, being accessible from anywhere, is versatile in any engagement. The cloud is the perfect location to host C2 infrastructure. It allows quick and programmable deployments that can be accessed from anywhere in the world. Some cloud providers will even support HTTPS, allowing for the quick spin up of a C2 without having to worry about purchasing and managing domains or certificates.
The popular choice for penetration testers is Amazon Web Services (AWS), a leader in the cloud space. Its services are fairly inexpensive and it offers an introductory free tier option.
Other viable cloud providers include the following:
Microsoft Azure: https://portal.azure.com
Google Cloud Platform: https://cloud.google.com
DigitalOcean: https://www.digitalocean.com
Linode: https://www.linode.com
Microsoft's Azure has a software as a service (SaaS) free tier feature that lets you deploy C2 automatically from a GitHub repository. It also provides HTTPS support out of the box, making it easier to hide C2 data from prying eyes and enabling it to blend in with normal user traffic.
Note
Always get permission (in writing!) from the cloud provider before conducting assessments using its infrastructure, even if it's something as simple as hosting a malicious JavaScript file on a temporary virtual machine.
Cloud internet service providers (ISPs) should have a form available for you to fill out that will detail an upcoming penetration test on their infrastructure. A testing window and contact information will likely need to be provided.
Whether we are using the cloud to house a C2 for an engagement or attacking applications hosted in the cloud, we should always notify the client of penetration testing - related activity.
Figure 1.7: A typical penetration test notification form
Consult the following resources for more information on penetration testing tools and techniques:
Penetration Testers Framework (PTF): https://github.com/trustedsec/ptf
BlackArch: https://blackarch.org
Burp Suite: https://portswigger.net/burp/
OWASP ZAP: https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project
Amazon Web Services: https://aws.amazon.com
Microsoft Azure: https://portal.azure.com
Google Cloud Platform: https://cloud.google.com
DigitalOcean: https://www.digitalocean.com
Linode: https://www.linode.com
Complete the following exercises to get better acquainted with the hacker toolset and the tools we'll be using throughout this book:
Download and install your preferred penetration testing distribution: Kali or BlackArch, or play around with PTF
Use Burp Suite Free or ZAP to intercept, inspect, and modify traffic to your favorite site
Create a free account on the cloud computing provider of your choice and use its free tier to launch a Linux virtual machine instance
In this chapter, we looked at tools, environments, and the bare minimum ROE we must follow during engagements. We stressed how important communication is and how critical it is to consider client privacy while testing. We are not the bad guys and we cannot operate with impunity. We've also gone over the clean - up process and it is vital that we leave no artifacts, unless otherwise requested by the client. Our leftover shells should not be the feature of a future breach.
We've also covered the penetration tester's toolkit; an all-in-one Linux distribution, Kali; and a couple of its alternatives. The more important piece to a web application hacker's toolkit is arguably the attack proxy, two of which we've highlighted: Burp Suite and ZAP. Finally, we've mentioned the cloud as an emerging useful tool for the web application tester.
The attacker's job will always be easier than that of the defender. Any professional hacker with experience in the corporate world will attest to this. The attacker needs just one weak link in the chain — even if that weakness is temporary — to own the environment completely.
Security is difficult to do right the first time and it is even more difficult to keep it close to the baseline as time passes. There are often resourcing issues, lack of knowledge, or wrong priorities, including simply making the organization profitable. Applications have to be useable — they must be available and provide feature enhancements to be useful. There never seems to be enough time to test the code properly, let alone to test it for security bugs.
Staff turnover can also lead to inexperienced developers shipping insufficiently-tested code. The security team is often stretched thin with daily incidents, let alone having the time to be bothered with secure code reviews. There is no silver bullet for security testing applications and there is rarely enough money in the budget. There are many pieces to this puzzle and many factors that act against a completely secure application and underlying infrastructure.
This is where the professional hacker, who understands these limitations, can shine. With shell access to a server, one can search for a potential privilege escalation exploit, try to get it working, and, after some trial and error, gain full access. Alternatively, one could take advantage of the fact that inter-server communication is a common sysadmin requirement. This means that connections between servers are either passwordless, or that the password is improperly stored somewhere close by. It's not uncommon to find unprotected private keys in globally-readable directories, allowing access to every other server in the infrastructure. Secure Shell (SSH) private keys, frequently used in automating SSH connections, are not password protected because password protecting a private key will break the automation script that is using it.
In upcoming chapters, we will use these unfortunate truths about application development and deployment to our advantage.
