Nmap: Network Exploration and Security Auditing Cookbook - Second Edition

Chapter 1. Nmap Fundamentals

In this chapter, we will cover the following recipes:

Building Nmap's source code
Finding live hosts in your network
Listing open ports on a target host
Fingerprinting OS and services running on a target host
Using NSE scripts against a target host
Reading targets from a file
Scanning an IP address ranges
Scanning random targets on the Internet
Collecting signatures of web servers
Monitoring servers remotely with Nmap and Ndiff
Crafting ICMP echo replies with Nping
Managing multiple scanning profiles with Zenmap
Running Lua scripts against a network connection with Ncat
Discovering systems with weak passwords with Ncrack
Launching Nmap scans remotely from a web browser using Rainmap Lite

Introduction

Network Mapper (Nmap) was originally released by Gordon Fyodor Lyon in the infamous Phrack magazine Vol 7 Issue 51 (https://nmap.org/p51-11.html). It is acclaimed today as one the best tools for network reconnaissance and security auditing in the information security industry. The first public version was introduced as an advanced port scanner along with a paper describing research on techniques for port discovery, but it has become so much more. It has evolved into an essential, fully featured tool that includes several other great subprojects, such as Ncrack, Ncat, Nping, Zenmap, and the Nmap Scripting Engine (all of them are available at https://nmap.org/). Nmap is described as follows in the official website:

"Nmap (Network Mapper) is a free and open source (license) utility for network discovery and security auditing. Many systems and network administrators also find it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics. It was designed to rapidly scan large networks, but works fine against single hosts. Nmap runs on all major computer operating systems, and official binary packages are available for Linux, Windows, and Mac OS X."

Other tools in the project were created to meet the specific needs of users. Nping (https://nmap.org/nping/) specializes in network packet crafting. Ncrack (https://nmap.org/ncrack/) focuses on network authentication cracking. Ncat (https://nmap.org/ncat/) is an enhanced version of Netcat and allows users to read, write, redirect, and modify network data. Zenmap (https://nmap.org/zenmap/) is a cross-platform GUI focused on usability. Finally, the Nmap Scripting Engine (https://nmap.org/book/nse.html) takes scanned information obtained from targets and provides an interface for users to script additional tasks.

Nmap's community is very active, so I encourage you to always keep up with the releases and latest patches. Announcements and discussions take place on the development mailing list, so if you would like to contribute to the project, I recommend you subscribe to it.

This first chapter is for newcomers. Starting with building Nmap, we will become familiar with all the tools of the Nmap project. In just a few recipes, you will learn how flexible and powerful Nmap really is, but as we move through chapters, we will go deep into the internals to learn not only how to use the tools but to extend them and create your own. The practical tasks chosen for this chapter will help you fingerprint local and remote systems, map networks, craft custom network packets, and even identify systems with weak passwords.

Building Nmap's source code

Throughout the following recipes, we will use the tools included with the Nmap project, so it is a good idea to install the latest versions now. This recipe will show how to download the latest copy of the source code from the development repositories and install Nmap and related tools in your UNIX-based system.

We always prefer working with the very latest stable version of the repository because precompiled packages take time to prepare and we may miss a patch or a new NSE script. The following recipe will show the process of configuring, building, and maintaining an up-to-date copy of the Nmap project in your arsenal.

Getting ready

Before continuing, you need to have a working Internet connection and access to a subversion client. Unix-based platforms come with a command-line client named subversion (svn). To check whether it's already installed in your system, just open a terminal and type the following command:

$ svn

If the command was not found, install svn using your favorite package manager or build it from source code. The instructions to build svn from source code are out of the scope of this book, but they are widely documented online. Use your favorite search engine to find specific instructions for your system.

When building Nmap, we will also need additional libraries such as the development definitions from OpenSSL or the make command. In Debian based systems, try the following command to install the missing dependencies:

#apt-get install libssl-dev autoconf make g++

Note that OpenSSL is optional, and Nmap can be built without it; however, Nmap will be crippled as it uses OpenSSL for functions related to multiprecision integers, hashing and encoding/decoding for service detection, and the Nmap Scripting Engine.

How to do it...

First, we need to grab a copy of the source code from the official repositories. To download the latest version of the development branch, we use the checkout (or co) command:

$svn co --username guest https://svn.nmap.org/nmap

Now you should see the list of downloaded files and the message Checked out revision <Revision number>. A new directory containing the source code is now available in your working directory. After we install the required dependencies, we are ready to compile Nmap with the standard procedure: configure, make, and make install. Go into the directory containing the source code and enter the following:

$./configure

If the configuration process completes successfully, you should see some nice ASCII art (it's selected randomly, so you might not necessarily see this one):

To compile Nmap, use make:

$make

Now you should see the binary nmap in your current working directory. Finally, to install Nmap on the system, execute make install with administrative privileges:

#make install

You should see the message NMAP SUCCESFULLY INSTALLED when the operation is complete.

How it works...

The SVN repository hosted at https://svn.nmap.org/nmap contains the latest stable version of Nmap and has world read access that allows anyone to grab a copy of the source code. We built the project from scratch to get the latest patches and features. The installation process described in this recipe also installed Zenmap, Ndiff, and Nping.

There's more...

The process of compiling Nmap is similar to compiling other Unix-based applications, but there are several compiled time variables that can be adjusted to configure the installation. Precompiles binaries are recommended for users who can't compile Nmap from source. Unix-based systems are recommended because of some Windows limitations described at https://nmap.org/book/inst-windows.html.

Experimental branches

If you want to try the latest creations of the development team, there is a folder named nmap-exp that contains several experimental branches of the project. The code stored in this folder is not guaranteed to work all the time as it is used as a sandbox until it is ready to be merged in production. The subversion URL of this folder is https://svn.nmap.org/nmap-exp/.

Updating your local working copy

The Nmap project is very active (especially during summer), so do not forget to update your copy regularly. If you keep a working copy of the svn repository, you may do this easily by executing the following commands inside that directory:

$svn up
$make
#make install

Customizing the building process

If you do not need the other Nmap utilities, such as Nping, Ndiff, or Zenmap, you may use different configure directives to omit their installation during the configuration step:

./configure --without-ndiff
./configure --without-zenmap
./configure --without-nping

For a complete list of configuration directives, use the --help command argument:

$./configure --help

Precompiled packages

Precompiled Nmap packages can be found for all major platforms at https://nmap.org/download.html for those who do not have access to a compiler. When working with precompiled packages, just make sure that you grab a fairly recent version to avoid missing important fixes or enhancements.

Finding live hosts in your network

Finding live hosts in your local network is a common task among penetration testers and system administrators to enumerate active machines on a network segment. Nmap offers higher detection rates over the traditional ping utility because it sends additional probes than the traditional ICMP echo request to discover hosts.

This recipe describes how to perform a ping scan with Nmap to find live hosts in a local network.

How to do it...

Launch a ping scan against a network segment using the following command:

#nmap -sn <target>

The results will include all the hosts that responded to any of the packets sent by Nmap during the ping scan; that is, the active machines on the specified network segment:

   Nmap scan report for 192.168.0.1 
   Host is up (0.0025s latency). 
   MAC Address: F4:B7:E2:0A:DA:18 (Hon Hai Precision Ind.) 
   Nmap scan report for 192.168.0.2 
   Host is up (0.0065s latency). 
   MAC Address: 00:18:F5:0F:AD:01 (Shenzhen Streaming Video Technology   
   Company Limited) 
   Nmap scan report for 192.168.0.3 
   Host is up (0.00015s latency). 
   MAC Address: 9C:2A:70:10:84:BF (Hon Hai Precision Ind.) 
   Nmap scan report for 192.168.0.8 
   Host is up (0.029s latency). 
   MAC Address: C8:02:10:39:54:D2 (LG Innotek) 
   Nmap scan report for 192.168.0.10 
   Host is up (0.0072s latency). 
   MAC Address: 90:F6:52:EE:77:E9 (Tp-link Technologies) 
   Nmap scan report for 192.168.0.11 
   Host is up (0.030s latency). 
   MAC Address: 80:D2:1D:2C:20:55 (AzureWave Technology) 
   Nmap scan report for 192.168.0.18 
   Host is up (-0.054s latency). 
   MAC Address: 78:31:C1:C1:9C:0A (Apple) 
   Nmap scan report for 192.168.0.22 
   Host is up (0.030s latency). 
   MAC Address: F0:25:B7:EB:DD:21 (Samsung Electro Mechanics) 
   Nmap scan report for 192.168.0.5 
   Host is up. 
   Nmap done: 256 IP addresses (9 hosts up) scanned in 27.86 seconds

Ping scans in Nmap may also identify MAC addresses and vendors if executed as a privileged user on local Ethernet networks.

How it works...

The Nmap option -sn disables port scanning, leaving the discovery phase enabled, which makes Nmap perform a ping sweep. Depending on the privileges, Nmap by default uses different techniques to achieve this task: sending a TCP SYN packet to port 443, TCP ACK packet to port 80 and ICMP echo and timestamp requests if executed as a privileged user, or a SYN packets to port 80 and 443 via the connect() syscall if executed by users who can't send raw packets. ARP/Neighbor Discovery is also enabled when scanning local Ethernet networks as privileged users. MAC addresses and vendors are identified from the ARP requests sent during the ARP/Neighbor Discovery phase.

There's more...

Nmap supports several host discovery techniques, and probes can be customized to scan hosts effectively even in the most restricted environments. It is important that we understand the internals of the supported techniques to apply them correctly. Now, let's learn more about host discovery with Nmap.

Tracing routes

Ping scans allows including trace route information of the targets. Use the Nmap option --traceroute to trace the route from the scanning machine to the target host:

#nmap -sn --traceroute google.com microsoft.com
   Nmap scan report for google.com (216.58.193.46) 
   Host is up (0.16s latency). 
   Other addresses for google.com (not scanned):   
   2607:f8b0:4012:805::200e 
   rDNS record for 216.58.193.46: qro01s13-in-f14.1e100.net 

   TRACEROUTE (using port 443/tcp) 
   HOP RTT       ADDRESS 
   1   1.28 ms   192.168.0.1 
   2   ... 
   3   158.85 ms 10.165.1.9 
   4   ... 5 
   6   165.50 ms 10.244.158.13 
   7   171.18 ms 10.162.0.254 
   8   175.33 ms 200.79.231.81.static.cableonline.com.mx 
       (200.79.231.81) 
   9   183.16 ms 10.19.132.97 
   10  218.60 ms 72.14.203.70 
   11  223.35 ms 209.85.240.177 
   12  242.60 ms 209.85.142.47 
   13  ... 
   14  234.79 ms 72.14.233.237 
   15  235.17 ms qro01s13-in-f14.1e100.net (216.58.193.46)  
   Nmap scan report for microsoft.com (23.96.52.53) 
   Host is up (0.27s latency). 
   Other addresses for microsoft.com (not scanned): 23.100.122.175      
   104.40.211.35 104.43.195.251 191.239.213.197  
   TRACEROUTE (using port 443/tcp) 
   HOP RTT       ADDRESS 
   -   Hops 1-9 are the same as for 216.58.193.46 
   10  183.27 ms 10.19.132.30 
   11  231.26 ms 206.41.108.25 
   12  236.77 ms ae5-0.atb-96cbe-1c.ntwk.msn.net (104.44.224.230) 
   13  226.22 ms be-3-0.ibr01.bn1.ntwk.msn.net (104.44.4.49) 
   14  226.89 ms be-1-0.ibr02.bn1.ntwk.msn.net (104.44.4.63) 
   15  213.92 ms be-3-0.ibr02.was05.ntwk.msn.net (104.44.4.26) 
   16  251.91 ms ae71-0.bl2-96c-1b.ntwk.msn.net (104.44.8.173) 
   17  ... 19 
   20  220.70 ms 23.96.52.53 
   Nmap done: 2 IP addresses (2 hosts up) scanned in 67.85 seconds

Running the Nmap Scripting Engine during host discovery

The Nmap Scripting Engine can be enabled during ping scans to obtain additional information. As with any other NSE script, its execution will depend on the hostrule specified. To execute a NSE script with ping scans, we simply use the Nmap option --script <file,folder,category>, the same way as we would normally call NSE scripts with port/service detection scans:

#nmap -sn --script dns-brute websec.mx
   Nmap scan report for websec.mx (54.210.49.18) 
   Host is up. 
   rDNS record for 54.210.49.18: ec2-54-210-49-18.compute-  
   1.amazonaws.com 

   Host script results: 
   | dns-brute:  
   |   DNS Brute-force hostnames:  
   |     ipv6.websec.mx - 54.210.49.18 
   |     web.websec.mx - 198.58.116.134 
   |     www.websec.mx - 54.210.49.18 
   |_    beta.websec.mx - 54.210.49.18

Another interesting NSE script to try when discovering live hosts in networks is the script broadcast-ping:

$ nmap -sn --script broadcast-ping 192.168.0.1/24 
   Pre-scan script results: 
   | broadcast-ping:  
   |   IP: 192.168.0.11  MAC: 80:d2:1d:2c:20:55 
   |   IP: 192.168.0.18  MAC: 78:31:c1:c1:9c:0a 
   |_  Use --script-args=newtargets to add the results as targets

Exploring more ping scanning techniques

Nmap supports several ping scanning techniques using different protocols. For example, the default ping scan command with no arguments (nmap -sn <target>) as a privileged user internally executes the -PS443 -PA80 -PE -PP options corresponding to TCP SYN to port 443, TCP ACK to port 80, and ICMP echo and timestamps requests.

In Chapter 2, Network Exploration, you will learn more about the following ping scanning techniques supported in Nmap:

-PS/PA/PU/PY [portlist]: TCP SYN/ACK, UDP or SCTP discovery to given ports
-PE/PP/PM: ICMP echo, timestamp, and netmask request discovery probes
-PO [protocol list]: IP protocol ping

Listing open ports on a target host

This recipe describes how to use Nmap to determine the port states on a remote host, a process used to identify running services commonly referred to as port scanning. This is one of the tasks Nmap excels at, so it is important to learn the essential Nmap options related to port scanning.

How to do it...

To launch a default scan, the bare minimum you need is a target. A target can be an IP address, a host name, or a network range:

$nmap scanme.nmap.org

The scan results will show all the host information obtained, such as IPv4 (and IPv6 if available) address, reverse DNS name, and interesting ports with service names. All listed ports have a state. Ports marked as opened are of special interest as they represent services running on the target host:

   Nmap scan report for scanme.nmap.org (45.33.32.156) 
   Host is up (0.16s latency). 
   Other addresses for scanme.nmap.org (not scanned):      
   2600:3c01::f03c:91ff:fe18:bb2f 
   Not shown: 995 closed ports 
   PORT      STATE    SERVICE  
   22/tcp    open     ssh 
   25/tcp    filtered smtp 
   80/tcp    open     http 
   9929/tcp  open     nping-echo 
   31337/tcp open     Elite 
   Nmap done: 1 IP address (1 host up) scanned in 333.35 seconds

How it works...

The basic default Nmap scan nmap <target> executes a simple port scan that returns a list of ports. In addition, it returns a service name from a database distributed with Nmap and the port state for each of the listed ports.

Nmap categorizes ports into the following states:

Open: Open indicates that a service is listening for connections on this port.
Closed: Closed indicates that the probes were received, but it was concluded that there was no service running on this port.
Filtered: Filtered indicates that there were no signs that the probes were received and the state could not be established. It also indicates that the probes are being dropped by some kind of filtering.
Unfiltered: Unfiltered indicates that the probes were received but a state could not be established.
Open/Filtered: This indicates that the port was filtered or open but the state could not be established.
Close/Filtered: This indicates that the port was filtered or closed but the state could not be established.

Even for this simplest port scan, Nmap does many things in the background that can be configured as well. Nmap begins by converting the hostname to an IPv4 address using DNS name resolution. If you wish to use a different DNS server, use --dns-servers <serv1[,serv2],...>, or use-n if you wish to skip this step, as follows:

$ nmap --dns-servers 8.8.8.8,8.8.4.4 scanme.nmap.org

Afterward, it performs a host discovery process to check whether the host is alive (see the Finding live hosts in your network recipe). To skip this step, use -Pn as follows:

$ nmap -Pn scanme.nmap.org

Nmap then converts the IPv4 or IPv6 address back to a hostname using a reverse DNS query. Use -n to skip this step, as follows:

$ nmap -n scanme.nmap.org

Finally, it launches either a SYN stealth scan or TCP connect scan depending on the user privileges.

There's more...

Port scanning is one of the most powerful features available, and it is important that we understand the different techniques and Nmap options that affect the scan behavior.

Privileged versus unprivileged

Running the simplest port scan command, nmap <target>, as a privileged user by default launches a SYN Stealth Scan, whereas unprivileged users that cannot create raw packets use the TCP Connect Scan technique. The difference between these two techniques is that TCP Connect Scan uses the high-level connect() system call to obtain the port state information, meaning that each TCP connection is fully completed and therefore slower. SYN Stealth Scans use raw packets to send specially crafted TCP packets to detect port states with a technique known as half open.

Scanning specific port ranges

Setting port ranges correctly during your scans will be very handy. You might be looking for infected machines that use a specific port to communicate or a specific service and do not really care about the rest. Narrowing down the port list also optimizes performance, which is very important when scanning multiple targets.

There are several accepted formats for the argument -p:

Port list:

# nmap -p80,443 localhost

Port range:

# nmap -p1-100 localhost

All ports:

# nmap -p- localhost

Specific ports by protocols:

# nmap -pT:25,U:53 <target>

Service name:

# nmap -p smtp <target>

Service name wildcards:

# nmap -p smtp* <target>

Only ports registered in Nmap services:

# nmap -p[1-65535] <target>

Selecting a network interface

Nmap attempts to automatically detect your active network interface; however, there are some situations where it will fail or perhaps we will need to select a different interface in order to test networking issues. To force Nmap to scan using a different network interface, use the argument -e:

#nmap -e <interface> <target>
#nmap -e eth2 scanme.nmap.org

You will need to set your network interface manually if you ever encounter the message WARNING: Unable to find appropriate interface for system route to.

More port scanning techniques

In this recipe, we talked about the two default scanning methods used in Nmap: SYN Stealth Scan and TCP Connect Scan. However, Nmap supports several more port scanning techniques. Use nmap -h or visit https://nmap.org/book/man-port-scanning-techniques.html to learn more about them.

Fingerprinting OS and services running on a target host

Version detection and OS detection are two of the most popular features of Nmap. Nmap is known for having the most comprehensive OS and service fingerprint databases. Knowing the platform (OS) and the exact version of a service is highly valuable for people looking for security vulnerabilities or monitoring their networks for any unauthorized changes. Fingerprinting services may also reveal additional information about a target, such as available modules and specific protocol information.

This recipe shows how to fingerprint the operating system and running services of a remote host using Nmap.

How to do it...

To enable service detection, add the Nmap option -sV to your port scan command:

$ nmap -sV <target>

The -sV option adds a table containing an additional column named VERSION, displaying the specific service version, if identified. Additional information will be enclosed in parentheses.

$ nmap -sV scanme.nmap.org
   Nmap scan report for scanme.nmap.org (45.33.32.156) 
   Host is up (1.4s latency). 
   Other addresses for scanme.nmap.org (not scanned):    
   2600:3c01::f03c:91ff:fe18:bb2f 
   Not shown: 994 closed ports 
   PORT      STATE    SERVICE    VERSION 
   22/tcp    open     ssh        OpenSSH 6.6.1p1 Ubuntu 2ubuntu2.3   
   (Ubuntu Linux; protocol 2.0) 
   25/tcp    filtered smtp 
   80/tcp    open     http       Apache httpd 2.4.7 ((Ubuntu)) 
   514/tcp   filtered shell 
   9929/tcp  open     nping-echo Nping echo 
   31337/tcp open     tcpwrapped 
   Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel 

   Service detection performed. Please report any incorrect results    at https://nmap.org/submit/ . 
   Nmap done: 1 IP address (1 host up) scanned in 137.71 seconds

To enable OS detection, add the Nmap option -O to your scan command. Note that OS detection requires Nmap to be run as a privileged user:

# nmap -O <target>

The result will now include OS information at the bottom of the port list:

# nmap -O scanme.nmap.org
   Nmap scan report for scanme.nmap.org (45.33.32.156) 
   Host is up (0.25s latency). 
   Other addresses for scanme.nmap.org (not scanned):    
   2600:3c01::f03c:91ff:fe18:bb2f 
   Not shown: 994 closed ports 
   PORT      STATE    SERVICE 
   22/tcp    open     ssh 
   25/tcp    filtered smtp 
   80/tcp    open     http 
   514/tcp   filtered shell 
   9929/tcp  open     nping-echo 
   31337/tcp open     Elite 
   Device type: WAP|general purpose|storage-misc 
   Running (JUST GUESSING): Actiontec embedded (99%), Linux          
   2.4.X|3.X (99%), Microsoft Windows 7|2012|XP (96%), BlueArc 
   embedded (91%) 
   OS CPE: cpe:/h:actiontec:mi424wr-gen3i cpe:/o:linux:linux_kernel    cpe:/o:linux:linux_kernel:2.4.37 cpe:/o:linux:linux_kernel:3.2      cpe:/o:microsoft:windows_7 cpe:/o:microsoft:windows_server_2012      cpe:/o:microsoft:windows_xp::sp3 cpe:/h:bluearc:titan_2100 
   Aggressive OS guesses: Actiontec MI424WR-GEN3I WAP (99%), DD-WRT    v24-sp2 (Linux 2.4.37) (98%), Linux 3.2 (98%), Microsoft Windows    7 or Windows Server 2012 (96%), Microsoft Windows XP SP3 (96%),      BlueArc Titan 2100 NAS device (91%) 
   No exact OS matches for host (test conditions non-ideal). 
   OS detection performed. Please report any incorrect results at      https://nmap.org/submit/ . 
   Nmap done: 1 IP address (1 host up) scanned in 114.03 seconds

How it works...

The Nmap option -sV enables service detection, which returns additional service and version information. Service detection is one of the most loved features of Nmap because it's very useful in many situations, such as identifying security vulnerabilities or making sure a service is running on a given port or a patch has been applied successfully.

This feature works by sending different probes defined in the nmap-service-probes file to the list of suspected open ports. The probes are selected based on how likely they can be used to identify a service.

Note

If you are interested in the inner workings, you can find very detailed documentation on how service detection mode works and how the file formats are used at https://nmap.org/book/vscan.html.

The -O option tells Nmap to attempt OS detection by sending several probes using the TCP, UDP, and ICMP protocols against opened and closed ports. OS detection mode is very powerful due to Nmap's user community, which obligingly contributes fingerprints that identify a wide variety of systems, including residential routers, IP webcams, operating systems, and many other hardware devices. It is important to note that OS detection requires raw packets, so Nmap need to be run with enough privileges.

Note

The complete documentation of the tests and probes sent during OS detection can be found at https://nmap.org/book/osdetect-methods.html.

Nmap uses the Common Platform Enumeration (CPE) as the naming scheme for service and operating system detection. This convention is used in the information security industry to identify packages, platforms, and systems.

There's more...

OS and version detection scan options can be configured thoroughly and are very powerful when tuning the performance. Let's learn about some additional Nmap options related to these scan modes.

Increasing version detection intensity

You can increase or decrease the amount of probes to use during version detection by changing the intensity level of the scan with the argument --version-intensity [0-9], as follows:

# nmap -sV --version-intensity 9 <target>

This Nmap option is incredibly effective against services running on nondefault ports due to configuration changes.

Aggressive detection mode

Nmap has a special flag to activate aggressive detection, namely -A. Aggressive mode enables OS detection (-O), version detection (-sV), script scanning (-sC), and traceroute (--traceroute). This mode sends a lot more probes, and it is more likely to be detected, but provides a lot of valuable host information. You can try aggressive detection with the following command:

# nmap -A <target>
   Nmap scan report for scanme.nmap.org (45.33.32.156) 
   Host is up (0.071s latency). 
   Other addresses for scanme.nmap.org (not scanned):     
   2600:3c01::f03c:91ff:fe18:bb2f 
   Not shown: 994 closed ports 
   PORT      STATE    SERVICE    VERSION 
   22/tcp    open     ssh        OpenSSH 6.6.1p1 Ubuntu 2ubuntu2.3       
   (Ubuntu Linux; protocol 2.0) 
   | ssh-hostkey:  
   |   1024 ac:00:a0:1a:82:ff:cc:55:99:dc:67:2b:34:97:6b:75 (DSA) 
   |   2048 20:3d:2d:44:62:2a:b0:5a:9d:b5:b3:05:14:c2:a6:b2 (RSA) 
   |_  256 96:02:bb:5e:57:54:1c:4e:45:2f:56:4c:4a:24:b2:57 (ECDSA) 
   25/tcp    filtered smtp 
   80/tcp    open     http       Apache httpd 2.4.7 ((Ubuntu)) 
   |_http-server-header: Apache/2.4.7 (Ubuntu) 
   |_http-title: Go ahead and ScanMe! 
   514/tcp   filtered shell 
   9929/tcp  open     nping-echo Nping echo 
   31337/tcp open     tcpwrapped 
   Device type: WAP|general purpose|storage-misc 
   Running (JUST GUESSING): Actiontec embedded (98%), Linux 2.4.X|3.X    
   (98%), Microsoft Windows 7|2012|XP (96%), BlueArc embedded (91%) 
   OS CPE: cpe:/h:actiontec:mi424wr-gen3i cpe:/o:linux:linux_kernel       
   cpe:/o:linux:linux_kernel:2.4.37 cpe:/o:linux:linux_kernel:3.2    
   cpe:/o:microsoft:windows_7 cpe:/o:microsoft:windows_server_2012   
   cpe:/o:microsoft:windows_xp::sp3 cpe:/h:bluearc:titan_2100 
   Aggressive OS guesses: Actiontec MI424WR-GEN3I WAP (98%), DD-WRT   
   v24-sp2 (Linux 2.4.37) (98%), Linux 3.2 (98%), Microsoft Windows 7   
   or Windows Server 2012 (96%), Microsoft Windows XP SP3 (96%),   
   BlueArc Titan 2100 NAS device (91%) 
   No exact OS matches for host (test conditions non-ideal). 
   Network Distance: 2 hops 
   Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel  
   TRACEROUTE (using port 80/tcp) 
   HOP RTT     ADDRESS 
   1   0.08 ms 192.168.254.2 
   2   0.03 ms scanme.nmap.org (45.33.32.156)  
   OS and Service detection performed. Please report any incorrect       
   results at https://nmap.org/submit/ . 
   Nmap done: 1 IP address (1 host up) scanned in 208.05 seconds

Configuring OS detection

In case OS detection fails, you can use the argument --osscan-guess to force Nmap to guess the operating system:

#nmap -O --osscan-guess <target>

To launch OS detection only when the scan conditions are ideal, use the argument --osscan-limit:

#nmap -O --osscan-limit <target>

OS detection in verbose mode

Try OS detection in verbose mode to see additional host information, such as the TCP and IP ID sequence number values:

#nmap -O -v <target>

The IP ID sequence number can be found under the label IP ID Sequence Generation. Note that incremental IP ID sequence numbers can be abused by port scanning techniques such as idle scan:

#nmap -O -v 192.168.0.1 
   Initiating Ping Scan at 11:14 
   Scanning 192.168.0.1 [4 ports] 
   Completed Ping Scan at 11:14, 0.00s elapsed (1 total hosts) 
   Initiating Parallel DNS resolution of 1 host. at 11:14 
   Completed Parallel DNS resolution of 1 host. at 11:14, 0.02s elapsed 
   Initiating SYN Stealth Scan at 11:14 
   Scanning 192.168.0.1 [1000 ports] 
   Discovered open port 80/tcp on 192.168.0.1 
   Completed SYN Stealth Scan at 11:14, 13.80s elapsed (1000 total 
   ports) 
   Initiating OS detection (try #1) against 192.168.0.1 
   Retrying OS detection (try #2) against 192.168.0.1 
   Nmap scan report for 192.168.0.1 
   Host is up (0.11s latency). 
   Not shown: 998 closed ports 
   PORT    STATE    SERVICE 
   80/tcp  open     http 
   514/tcp filtered shell 
   Device type: WAP|general purpose|storage-misc 
   Running (JUST GUESSING): Actiontec embedded (99%), Linux 2.4.X|3.X   
   (99%), Microsoft Windows 7|2012|XP (96%), BlueArc embedded (91%) 
   OS CPE: cpe:/h:actiontec:mi424wr-gen3i cpe:/o:linux:linux_kernel   
   cpe:/o:linux:linux_kernel:2.4.37 cpe:/o:linux:linux_kernel:3.2   
   cpe:/o:microsoft:windows_7 cpe:/o:microsoft:windows_server_2012 
   cpe:/o:microsoft:windows_xp::sp3 cpe:/h:bluearc:titan_2100 
   Aggressive OS guesses: Actiontec MI424WR-GEN3I WAP (99%), DD-WRT      
   v24-sp2 (Linux 2.4.37) (98%), Linux 3.2 (97%), 
    Microsoft Windows 7 or Windows Server 2012 (96%), Microsoft  
    Windows XP SP3 (96%),       
   BlueArc Titan 2100 NAS device (91%) 
   No exact OS matches for host (test conditions non-ideal). 
   TCP Sequence Prediction: Difficulty=259 (Good luck!) 
   IP ID Sequence Generation: Incremental 

   Read data files from: /usr/local/bin/../share/nmap 
   OS detection performed. Please report any incorrect results at   
   https://nmap.org/submit/ . 
   Nmap done: 1 IP address (1 host up) scanned in 32.40 seconds 
              Raw packets sent: 1281 (59.676KB) | Rcvd: 1249 (50.520KB)

Submitting new OS and service fingerprints

Nmap's accuracy comes from a database that has been collected over the years through user submissions. It is very important that we help keep this database up to date. Nmap will let you know when you can contribute to the project by submitting an unidentified operating system, device, or service.

Please take the time to submit your contributions, as Nmap's detection capabilities come directly from the databases. Visit https://nmap.org/cgi-bin/submit.cgi? to submit new fingerprints or corrections.

Using NSE scripts against a target host

The Nmap project introduced a feature named Nmap Scripting Engine that allows users to extend the capabilities of Nmap via Lua scripts. NSE scripts are very powerful and have become one of Nmap's main strengths, performing tasks from advanced version detection to vulnerability exploitation. The variety of scripts available (more than 500) help users perform a wide range of tasks using the target information obtained from scans.

The following recipe describes how to run NSE scripts, and the different options available to configure its execution.

How to do it...

Enable script scan using the Nmap option -sC. This mode will select all NSE scripts belonging to the default category and execute them against our targets:

$nmap -sC <target>
$nmap -sC scanme.nmap.org
   Nmap scan report for scanme.nmap.org (45.33.32.156) 
   Host is up (0.14s latency). 
   Other addresses for scanme.nmap.org (not scanned):      
   2600:3c01::f03c:91ff:fe18:bb2f 
   Not shown: 995 closed ports 
   PORT      STATE    SERVICE 
   22/tcp    open     ssh 
   | ssh-hostkey:  
   |   1024 ac:00:a0:1a:82:ff:cc:55:99:dc:67:2b:34:97:6b:75 (DSA) 
   |   2048 20:3d:2d:44:62:2a:b0:5a:9d:b5:b3:05:14:c2:a6:b2 (RSA) 
   |_  256 96:02:bb:5e:57:54:1c:4e:45:2f:56:4c:4a:24:b2:57 (ECDSA) 
   25/tcp    filtered smtp 
   80/tcp    open     http 
   |_http-title: Go ahead and ScanMe! 
   9929/tcp  open     nping-echo 
   31337/tcp open     Elite 
   Nmap done: 1 IP address (1 host up) scanned in 24.42 seconds

In this case, the results included the output of the ssh-hostkey and http-title scripts. The number of scripts executed depends on the host or port rules of the scripts.

How it works...

The Nmap option -sC enables script scan mode, which tells Nmap to select the default scripts and execute them if the host or port rule matches.

NSE scripts are divided into the following categories:

auth: This category is for scripts related to user authentication
broadcast: This is a very interesting category of scripts that use broadcast petitions to gather information
brute: This category is for scripts that help conduct brute-force password auditing
default: This category is for scripts that are executed when a script scan is executed (-sC)
discovery: This category is for scripts related to host and service discovery.
dos: This category is for scripts related to denial of service attacks
exploit: This category is for scripts that exploit security vulnerabilities
external: This category is for scripts that depend on a third-party service
fuzzer: This category is for NSE scripts that are focused on fuzzing
intrusive: This category is for scripts that might crash something or generate a lot of network noise; scripts that system administrators may consider intrusive belong to this category
malware: This category is for scripts related to malware detection
safe: This category is for scripts that are considered safe in all situations
version: This category is for scripts that are used for advanced versioning
vuln: This category is for scripts related to security vulnerabilities

There's more...

Let's learn about some Nmap options that are required to customize the Nmap Scripting Engine. Some scripts require to be configured correctly, so it is important that we are familiar with all the Nmap Scripting Engine options.

NSE script arguments

The --script-args flag is used to set the arguments of NSE scripts. For example, if you would like to set the useragent HTTP library argument, you would use the following:

$ nmap --script http-title --script-args http.useragent="Mozilla 999" <target>

You can also use aliases when setting the arguments for NSE scripts. For example, you have the following code:

$ nmap -p80 --script http-trace --script-args path <target>

Instead of the preceding code, you can use the following one:

$ nmap -p80 --script http-trace --script-args http-trace.path <target>

Script selection

Users may select specific scripts when scanning using the Nmap option --script <filename or path/folder/category/expression>:

$nmap --script <filename or path/folder/category/expression> <target>

For example, the command to run the NSE script dns-brute is as follows:

$nmap --script dns-brute <target>

The Nmap Scripting Engine also supports the execution of multiple scripts simultaneously:

$ nmap --script http-headers,http-title scanme.nmap.org
   Nmap scan report for scanme.nmap.org (74.207.244.221) 
   Host is up (0.096s latency). 
   Not shown: 995 closed ports 
   PORT     STATE    SERVICE 
   22/tcp   open     ssh 
   25/tcp   filtered smtp 
   80/tcp   open     http 
   | http-headers: 
   |   Date: Mon, 24 Oct 2011 07:12:09 GMT 
   |   Server: Apache/2.2.14 (Ubuntu) 
   |   Accept-Ranges: bytes 
   |   Vary: Accept-Encoding 
   |   Connection: close 
   |   Content-Type: text/html 
   |   
   |_  (Request type: HEAD) 
   |_http-title: Go ahead and ScanMe! 
   646/tcp  filtered ldp 
   9929/tcp open     nping-echo

In addition, NSE scripts can be selected by category, expression, or folder:

Run all the scripts in the vuln category:

$ nmap -sV --script vuln <target>

Run the scripts in the version or discovery categories:

$ nmap -sV --script="version,discovery" <target>

Run all the scripts except for the ones in the exploit category:

$ nmap -sV --script "not exploit" <target>

Run all HTTP scripts except http-brute and http-slowloris:

$ nmap -sV --script "(http-*) and not(http-slowloris or http-brute)" <target>

Expressions are very handy as they allow fine-grained script selection, as shown in the preceding example.

Debugging NSE scripts

To debug NSE scripts, use --script-trace. This enables a stack trace of the executed script to help you debug the script execution. Remember that sometimes you may need to increase the debugging level with the -d[1-9] flag to get to the bottom of the problem:

$ nmap -sC --script-trace <target>
$ nmap --script http-headers --script-trace scanme.nmap.org
   NSOCK INFO [18.7370s] nsock_trace_handler_callback(): Callback:   
   CONNECT SUCCESS for EID 8 [45.33.32.156:80] 
   NSE: TCP 192.168.0.5:47478 > 45.33.32.156:80 | CONNECT 
   NSE: TCP 192.168.0.5:47478 > 45.33.32.156:80 | 00000000: 
   48 45 41 44 20 2f 20 48 54 54 50 2f 31 2e 31 0d HEAD / HTTP/1.1  
   00000010: 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f     
   Connection: clo 
   00000020: 73 65 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 se  
   User- Agent:  
   00000030: 4d 6f 7a 69 6c 6c 61 2f 35 2e 30 20 28 63 6f 6d   
   Mozilla/5.0 (com 
   00000040: 70 61 74 69 62 6c 65 3b 20 4e 6d 61 70 20 53 63 patible;   
   Nmap Sc 
   00000050: 72 69 70 74 69 6e 67 20 45 6e 67 69 6e 65 3b 20 ripting   
   Engine;  
   00000060: 68 74 74 70 73 3a 2f 2f 6e 6d 61 70 2e 6f 72 67   
   https://nmap.org 
   00000070: 2f 62 6f 6f 6b 2f 6e 73 65 2e 68 74 6d 6c 29 0d       
   /book/nse.html)  
   00000080: 0a 48 6f 73 74 3a 20 73 63 61 6e 6d 65 2e 6e 6d  Host: 
   scanme.nm 
   00000090: 61 70 2e 6f 72 67 0d 0a 0d 0a                   ap.org     
   [Output removed to save space]Nmap scan report for scanme.nmap.org   
   (45.33.32.156) 
   Host is up (0.14s latency). 
   Other addresses for scanme.nmap.org (not scanned):    
   2600:3c01::f03c:91ff:fe18:bb2f 
   Not shown: 995 closed ports 
   PORT      STATE    SERVICE 
   22/tcp    open     ssh 
   25/tcp    filtered smtp 
   80/tcp    open     http 
   | http-headers:  
   |   Date: Sun, 24 Apr 2016 19:52:13 GMT 
   |   Server: Apache/2.4.7 (Ubuntu) 
   |   Accept-Ranges: bytes 
   |   Vary: Accept-Encoding 
   |   Connection: close 
   |   Content-Type: text/html 
   |    
   |_  (Request type: HEAD) 
   9929/tcp  open     nping-echo 
   31337/tcp open     Elite 

   Nmap done: 1 IP address (1 host up) scanned in 18.89 seconds

Adding new scripts

There will be occasions where you will want to try scripts not included officially with Nmap. To test new scripts, you simply need to copy them to your /scripts inside your Nmap directory and run the following command to update the script database:

# nmap --script-updatedb

After updating the script database, you simply need to select them, as you would normally do with the --script option. In addition, you may execute scripts without including them in the database by setting a relative or absolute script path as the argument:

# nmap --script /root/loot/nonofficial.nse <target>

The https://secwiki.org/w/Nmap/External_Script_Library Wiki page attempts to keep track of all scripts that for different reasons could not get included officially with Nmap. I recommend you visit it as there are some great scripts in there.

Scanning an IP address ranges

Very often, penetration testers and system administrators need to scan not a single machine but a range of hosts. Nmap supports IP address ranges in different formats, and it is essential that we know how to deal with them.

This recipe explains how to work with IP address ranges when scanning with Nmap.

How to do it...

Open your terminal and enter the following command:

$ nmap <IP address range>

For example, to scan from 192.168.1.0 to 192.168.1.255 use the following command:

$ nmap 192.168.1.0-255

Alternatively, you can use any of the following notations:

$ nmap 192.168.*
$ nmap 192.168.0/24
$ nmap 192.168.1.0 192.168.1.1 192.168.1.2 ... 192.168.1.254 192.168.1.255

How it works...

Nmap supports several target formats that allows users to work with IP address ranges. The most common type is when we specify the target's IP or host, but it also supports the reading of targets from files, ranges, and we can even generate a list of random targets.

Any arguments that are not valid options are read as targets by Nmap. This means that we can tell Nmap to scan more than one range in a single command, as shown in the following command:

# nmap -p25,80 -O -T4 192.168.1.1/24 scanme.nmap.org/24

There are several ways that we can handle IP ranges in Nmap:

Multiple host specification
Octet range addressing (they also support wildcards)
CIDR notation

To scan IP addresses 192.168.1.1, 192.168.1.2, and 192.168.1.3, the following command can be used:

$ nmap 192.168.1.1 192.168.1.2 192.168.1.3

We can also specify octet ranges using -. For example, to scan hosts 192.168.1.1, 192.168.1.2, and 192.168.1.3, we could use the expression 192.168.1.1-3, as shown in the following command:

$ nmap 192.168.1.1-3

Octect range notation also supports wildcards, so we could scan from 192.168.1.0 to 192.168.1.255 with the expression 192.168.1.*:

$ nmap 192.168.1.*

The CIDR notation can also be used when specifying targets. The CIDR notation consists of an IP address and a suffix. The most common network suffixes used are /8, /16, /24, and /32. To scan the 256 hosts in 192.168.1.0-255 using the CIDR notation, the following command can be used:

$ nmap 192.168.1.0/24

There's more...

In addition, you may exclude the hosts from the ranges by specifying the parameter the --exclude option, as shown:

$ nmap 192.168.1.1-255 --exclude 192.168.1.1
$ nmap 192.168.1.1-255 --exclude 192.168.1.1,192.168.1.2

Otherwise, you can write your exclusion list in a file and read it with--exclude-file:

$ cat dontscan.txt
   192.168.1.1
   192.168.1.254
$ nmap --exclude-file dontscan.txt 192.168.1.1-255

CIDR notation

The Classless Inter-domain Routing (CIDR) notation (pronounced cider) is a compact method for specifying IP addresses and their routing suffixes. This notation gained popularity due to its granularity when compared with classful addressing because it allows subnet masks of variable length.

The CIDR notation is specified by an IP address and network suffix. The network or IP suffix represent the number of network bits. IPv4 addresses are 32 bit, so the network can be between 0 and 32. The most common suffixes are /8, /16, /24, and /32.

To visualize it, take a look at the following CIDR-to-Netmask conversion table:

CIDR	Netmask
/8	255.0.0.0
/16	255.255.0.0
/24	255.255.255.0
/32	255.255.255.255

For example, 192.168.1.0/24 represents the 256 IP addresses from 192.168.1.0 to 192.168.1.255. And 50.116.1.121/8 represents all the IP addresses between 50.0-255.0-255.0-255. The network suffix /32 is also valid and represents a single IP.

Scanning random targets on the Internet

Nmap supports a very interesting feature that allows us to run scans against random targets on the Internet. Although it is not recommended (and probably not legal) to do aggressive scans blindly, this is very useful when conducting research that needs a sample of random hosts.

This recipe shows you how to generate random hosts as targets for your Nmap scans.

How to do it...

To generate a random target list of n hosts, use the following Nmap command:

$ nmap -iR <n>

For example, to generate a list of 100 targets, we use the following command:

$ nmap -iR 100

Now, let's check how common is ICMP in remote servers. Let's launch a ping scan against three random targets:

$ nmap -sn -iR 3
   Nmap scan report for host86-190-227-45.wlms-broadband.com   
   (86.190.227.45) 
   Host is up (0.000072s latency). 
   Nmap scan report for 126.182.245.207 
   Host is up (0.00023s latency). 
   Nmap scan report for 158.sub-75-225-31.myvzw.com (75.225.31.158) 
   Host is up (0.00017s latency). 
   Nmap done: 3 IP addresses (3 hosts up) scanned in 0.78 seconds

How it works...

The argument -iR 100 tells Nmap to generate 100 external IP addresses and use them as targets in the specified scan. This target assignment can be used with any combination of scan flags.

While this is a useful feature for conducting Internet research, I recommend you to be careful with this flag. Nmap does not have control over the external IP addresses it generates; this means that inside the generated list could be a critical machine that is being heavily monitored. To avoid getting into trouble, use this feature wisely.

There's more...

To tell Nmap to generate an unlimited number of IPs and hence run indefinitely, set the argument -iR to 0 using the following command:

$ nmap -iR 0

For example, to find random NFS shares online, you could use the following command:

$ nmap -p2049 --open -iR 0

Legal issues with port scanning

Port scanning without permission is not very welcome, and it is even illegal in some countries. I recommend you to research your local laws to find out what you are permitted to do and if port scanning is frowned upon in your country. You also need to consult with your ISP as they may have their own rules on the subject.

The official documentation of Nmap has an amazing write-up about the legal issues involved with port scanning, available at https://nmap.org/book/legal-issues.html. I recommend that everyone considering doing Internet-wide research scanning reads it.

Collecting signatures of web servers

Nmap is an amazing tool for information gathering, and the variety of tasks that can be done with the Nmap Scripting Engine is simply remarkable. The popular service ShodanHQ (https://www.shodan.io/) offers a database of HTTP banners, which is useful for analyzing the impact of vulnerabilities. Its users can find out the number of devices that are online by country, which are identified by their service banners. ShodanHQ uses its own built-in house tools to gather its data, but Nmap can easily be used for this task.

In the following recipe, we will see how to scan indefinitely for web servers, and collect their HTTP headers with Nmap.

How to do it...

Open your terminal and enter the following command:

$ nmap -p80,443 -Pn -T4 --open --script http-headers,http-title,ssl-cert --script-args http.useragent="A friendly web crawler (http://calderonpale.com)",http-headers.useget -oX random-webservers.xml -iR 0

This command will launch an instance of Nmap that will run indefinitely, looking for web servers in port 80 and 443 and then save the output to random-webservers.xml. Each host that has port 80 or 443 open will return something like the following:

   Nmap scan report for XXXX 
   Host is up (0.23s latency). 
   PORT   STATE SERVICE 
   80/tcp open  http 
   |_http-title: Protected Object 
   | http-headers: 
   |   WWW-Authenticate: Basic realm="TD-8840T" 
   |   Content-Type: text/html 
   |   Transfer-Encoding: chunked 
   |   Server: RomPager/4.07 UPnP/1.0 
   |   Connection: close 
   |   EXT: 
   | 
   |_  (Request type: GET)

How it works...

The following command will tell Nmap to only check port 80 or 443 (-p80,443), without ping (-Pn), and to use the aggressive timing template (-T4). If port 80 or 443 is open, Nmap will run the NSE scripts http-title, http-headers, and ssl-cert(--script http-headers,http-title,ssl-cert) to collect server headers and web server title; if HTTPS is detected, we will also extract information from SSL certificates if available:

$nmap -p80 -Pn -T4 --open --script http-headers,http-title --script-args http.useragent="A friendly web crawler (http://calderonpale.com)",http-headers.useget -oX random-webservers.xml -iR 0

The script arguments that are passed are used to set the HTTP user agent in the requests (--script-args http.useragent="A friendly web crawler (http://calderonpale.com)") and use a GET request to retrieve the HTTP headers (--script-args http-headers.useget).

Finally, the argument -iR 0 tell Nmap to generate external IP addresses indefinitely and save the results in a file in XML format (-oX random-webservers.xml).

There's more...

Nmap's HTTP library has cache support, but if you are planning to scan many hosts, you need to consider your cache file. The cache is stored in a temporary file that grows with each new request. If this file starts to get too big, cache lookups start to take a considerable amount of time.

You can disable the cache system of the HTTP library by setting the http-max-cache-size=0 library argument, as shown in the following command:

$ nmap -p80 --script http-headers --script-args http-max-cache-size=0  -iR 0

Note

The HTTP NSE library is highly configurable. Read Appendix A, HTTP, HTTP Pipelining, and Web Crawling Configuration Options, to learn more about the advanced options available.

Monitoring servers remotely with Nmap and Ndiff

Using tools from the Nmap project we can set up a simple but powerful monitoring system. Because our monitoring system will depend on Nmap, we can monitor any information Nmap can gather. To detect changes on the network, we will need to compare the results of two scans: the base or known good state and the last results obtained. Now it is the perfect time to introduce Ndiff.

Ndiff was designed to address the issues of using the traditional diff command with two XML scan results. It compares files by removing false positives and producing a more readable output, which is perfect for anyone who needs to keep track of the scan results.

This recipe describes how to use bash scripting, cron, Nmap, and Ndiff to set up a monitoring system that alerts the user by e-mail if changes are detected in a network.

Getting ready

In this recipe, we assume the system has been configured to send mail via the mail command. If you would like to change the notification method, you simply need to update the bash script. You could use curl to POST data to your favorite social network or run a script that restarts the service. The possibilities are endless.

How to do it...

To setup a simple monitoring system with Nmap, we are going to need to do a few things:

Create the directory /usr/local/share/nmap-mon/ directory (or whatever location you prefer) to store all the files required for our monitoring system.
Scan your targets and save the result in XML format in the directory that you just created:

# nmap -oX base_results.xml -sV -Pn <target>

The resulting file base_results.xml file will be used as your base file, meaning that it should reflect the known good versions and ports.

Create the file nmap-mon.sh file in the directory you created earlier and paste the following code:

#!/bin/bash  
#Bash script to email admin when changes are detected in a network using Nmap and Ndiff.  
#  
#Don't forget to adjust the CONFIGURATION variables.  
#Paulino Calderon <calderon@websec.mx>  
#  
#CONFIGURATION  
#  
NETWORK="YOURTARGET"  
ADMIN=YOUR@EMAIL.COM  
NMAP_FLAGS="-n -sV -Pn -p- -T4"  
BASE_PATH=/usr/local/share/nmap-mon/  
BIN_PATH=/usr/local/bin/  
BASE_FILE=base.xml  
NDIFF_FILE=ndiff.log  
NEW_RESULTS_FILE=newscanresults.xml  
BASE_RESULTS="$BASE_PATH$BASE_FILE"  
NEW_RESULTS="$BASE_PATH$NEW_RESULTS_FILE"  
NDIFF_RESULTS="$BASE_PATH$NDIFF_FILE"   
if [ -f $BASE_RESULTS ]  
then  
  echo "Checking host $NETWORK"  
  ${BIN_PATH}nmap -oX $NEW_RESULTS $NMAP_FLAGS $NETWORK  
  ${BIN_PATH}ndiff $BASE_RESULTS $NEW_RESULTS > $NDIFF_RESULTS  
  if [ $(cat $NDIFF_RESULTS | wc -l) -gt 0 ]  
  then  
    echo "Network changes detected in $NETWORK"  
    cat $NDIFF_RESULTS  
    echo "Alerting admin $ADMIN"  
    mail -s "Network changes detected in $NETWORK" $ADMIN < $NDIFF_RESULTS  
  fi  
fi

Update the configuration values according to your system:

NETWORK="YOURTARGET"  
ADMIN=YOUR@EMAIL.COM  
NMAP_FLAGS="-sV -Pn -p- -T4"  
BASE_PATH=/usr/local/share/nmap-mon/  
BIN_PATH=/usr/local/bin/  
BASE_FILE=base.xml  
NDIFF_FILE=ndiff.log  
NEW_RESULTS_FILE=newscanresults.xml

Make nmap-mon.sh executable by entering the following command:

# chmod +x /usr/local/share/nmap-mon/nmap-mon.sh

Now run the nmap-mon.sh script to make sure it is working correctly.

# /usr/local/share/nmap-mon/nmap-mon.sh

Launch your crontab editor to execute the script periodically automatically:

# crontab -e

Add the following command:

0 * * * * /usr/local/share/nmap-mon/nmap-mon.sh

You should now receive e-mail alerts when Ndiff detects a change in your network.

How it works...

Ndiff is a tool for comparing two Nmap scans. Think about the traditional diff but for Nmap scan reports. With some help from bash and cron, we set up a task that is executed at regular intervals to scan our network and compare our current state with an older state, to identify the differences between them. We used some basic bash scripting to execute our monitoring scan and then executed Ndiff to compare the results:

  if [ $(cat $NDIFF_RESULTS | wc -l) -gt 0 ]  
  then  
    echo "Network changes detected in $NETWORK"  
    cat $NDIFF_RESULTS  
    echo "Alerting admin $ADMIN"  
    mail -s "Network changes detected in $NETWORK" $ADMIN < $NDIFF_RESULTS  
  fi

There's more...

You can adjust the interval between scans by modifying the cron line:

0 * * * * /usr/local/share/nmap-mon/nmap-mon.sh

To update your base file, you simply need to overwrite your base file located at /usr/local/share/nmap-mon/. Remember that when we change the scan parameters to create our base file, we need to update them in nmap-mon.sh too.

Monitoring specific services

To monitor some specific service, you need to update the scan parameters in nmap-mon.sh:

NMAP_FLAGS="-sV -Pn"

For example, if you would like to monitor a web server, you may use the following parameters:

NMAP_FLAGS="-sV --script http-google-safe -Pn -p80,443"

These parameters set port scanning only to ports 80 and 443, and in addition, these parameters include the http-google-safe script to check whether your web server has been marked as malicious by the Google safe browsing service.

Crafting ICMP echo replies with Nping

Nping is a utility designed to ease the process of crafting network packets. It is very useful to debug and troubleshoot network communications and perform traffic analysis.

This recipe will introduce Nping and go over the process of crafting and transmitting custom ICMP packets.

How to do it...

Let's say that we want to respond to an ICMP echo request packet with an echo reply using Nping. Consider that the first ICMP echo request packet has a source IP of 192.168.0.10 with an ICMP ID of 520, and the data string was the word ping. With that information, we can craft the reply with the following command:

#nping --icmp -c 1 --icmp-type 0 --icmp-code 0 --source-ip192.168.0.5 --dest-ip 192.168.0.10 --icmp-id 520 --icmp-seq 0--data-string 'ping'

In the output, you should see the sent ICMP echo reply packet with the values taken from the ICMP echo request packets:

   SENT (0.0060s) ICMP [192.168.0.5 > 192.168.0.10 Echo reply    
   (type=0/code=0) id=520 seq=0] IP [ttl=64 id=10898 iplen=32 ] 
   Max rtt: N/A | Min rtt: N/A | Avg rtt: N/A 
   Raw packets sent: 1 (32B) | Rcvd: 0 (0B) | Lost: 1 (100.00%) 
   Nping done: 1 IP address pinged in 1.01 seconds

How it works...

Nping allows configuring the values of most fields in TCP, UDP, ARP, and ICMP packets easily. The following command will send an ICMP echo reply packet with the values obtained from the ICMP echo request packet:

#nping --icmp -c 1 --icmp-type 0 --icmp-code 0 --source-ip192.168.0.5 --dest-ip 192.168.0.10 --icmp-id 520 --icmp-seq 0 --data-string 'ping'

Let's break it down by its arguments:

--icmp: This sets ICMP as the protocol to use.
-c 1: Packet count. Send only one packet.
--icmp-type 0 --icmp-code 0: This sets ICMP type and code. This type corresponds to an echo reply message.
--source-ip 192.168.0.5 --dest-ip 192.168.0.10: This sets the source and destination IP address.
--icmp-id 520: This sets the ICMP identifier of the request packet.
--icmp-seq 0: This sets the ICMP Sequence number.
--data-string 'ping': This sets the data string.

There's more...

Nping can set most fields in TCP, UDP, ARP, and ICMP packets via arguments but offers a lot more customization than we offer. In addition to the interesting timing and performance options, Nping supports a mode named echo that is handy when troubleshooting firewall or routing issues. I highly recommend you go over the documentation at https://nmap.org/nping/ to become familiar with this powerful tool and the scenarios where it can be handy.

Managing multiple scanning profiles with Zenmap

Scanning profiles are a combination of Nmap options and arguments that can be used to save time when launching Nmap scans.

This recipe is about adding, editing, and deleting a scanning profile in Zenmap.

How to do it...

Let's add a new profile for scanning web servers:

Launch Zenmap.
Click on Profile on the main toolbar.
Click on New Profile or Command (Ctrl + P). The Profile Editor will be launched.
Enter a profile name and a description on the Profile tab.
Enable Version detection and select TCP connect scan (-sT) in the Scan tab.
Enable Don't ping before scanning (-Pn) in the Ping tab.
Enable the following scripts on the Scripting tab:
- hostmap-ip2hosts
- http-apache-negotiation
- http-apache-server
- http-auth-finder
- http-backup-finder
- http-config-backup
- http-cors
- http-cross-domain-policy
- http-csrf
- http-default-accounts
- http-devframework
- http-dombased-xss
- http-enum
- http-exif-spider
- http-favicon
- http-git
- http-headers
- http-iis-short-name-brute
- http-methods
- http-mobileversion-checker
- http-ntlm-info
- http-open-proxy
- http-open-redirect
- http-trace
- http-php-version
- http-phpself-xss
- http-robots.txt
- http-server-header
- http-shellshock
- http-svn-info
- http-title
Next, go to the Target tab and click on Ports to scan (-p) and enter 80, 443.

Save your changes by clicking on Save Changes:

Your new scanning profile should be available on the Profile drop-down menu.

How it works...

After using the editor to create our profile, we are left with the following Nmap command:

$ nmap -sT -sV -p 80,443 -T4 -v -Pn --script hostmap-ip2hosts,http-apache-negotiation,http-apache-server-status,http-auth-finder,http-backup-finder,http-config-backup,http-cors,http-cross-domain-policy,http-csrf,http-default-accounts,http-devframework,http-dombased-xss,http-exif-spider,http-git,http-headers,http-iis-short-name-brute,http-methods,http-ntlm-info,http-open-proxy,http-open-redirect,http-phpself-xss,http-robots.txt,http-server-header,http-shellshock,http-svn-info,http-title,http-waf-detect <target>

Using the Profile wizard, we have enabled service scanning (-sV), set the scanning ports to 80 and 443, configure ping options (-Pn), and select a bunch of HTTP-related scripts to gather as much information as possible from this web server. We now have this command saved for our scanning activities against new targets in the future.

There's more...

Customizing scan profiles can be done through the user interface. Default scanning profiles can be used as templates when creating new ones. Let's review how we work with the scanning profiles.

Zenmap scanning profiles

The predefined Zenmap scanning profiles help newcomers familiarize themselves with Nmap. I recommend you to analyze them to understand the scanning techniques available in Nmap, along with some useful combinations of its options:

Intense scan: nmap -T4 -A -v
Intense scan plus UDP: nmap -sS -sU -T4 -A -v
Intense scan, all TCP ports: nmap -p 1-65535 -T4 -A -v
Intense scan, no ping: nmap -T4 -A -v -Pn
Ping scan: nmap -sn
Quick scan: nmap -T4 -F
Quick scan plus: nmap -sV -T4 -O -F -version-light
Quick traceroute: nmap -sn -traceroute
Regular scan: nmap
Slow comprehensive scan: nmap -sS -sU -T4 -A -v -PE -PP -PS80,443 -PA3389 -PU40125 -PY -g 53 --script default or discovery and safe

Note

You can find more scanning profiles at https://github.com/cldrn/rainmap-lite/wiki/Scanning-profiles.

Editing or deleting a scan profile

To edit or delete a scan profile, you need to select the entry you wish to modify from the Profile drop-down menu. Click on Profile on the main toolbar and select Edit Selected Profile (Ctrl + E).

The editor will be launched allowing you to edit or delete the selected profile.

Running Lua scripts against a network connection with Ncat

Ncat allows users to read, write, redirect, and modify network data in some very interesting ways. Think about it as an enhanced version of the traditional tool netcat. Ncatoffers the possibility of running external commands in different ways once a connection has been established successfully. One way is with the help of Lua scripts that act as programs and allow users to perform any task they wish.

The following recipe will show you how to run a HTTP server contained in a Lua script with Ncat.

How to do it...

Running Lua scripts against network connections in Ncat is very straightforward; just use the --lua-exec option to point to the Lua script you want to execute and the listening port or host to connect:

$ncat --lua-exec <path to Lua script> --listen 80

To start a web server with Ncat, locate the httpd.lua script inside your Ncat's script folder and use the command:

$ncat --lua-exec /path/to/httpd.lua --listen 8080 --keep-open

Ncat will start listening on port 8080 and execute the Lua program specified on connection. You may verify that the script is running correctly by pointing a web browser to that direction and checking whether the Got a request for message appears on the output.

How it works...

If you have ever used netcat, you will be familiar with Ncat. Similarly, Ncat can be put into listening (--listen) and connect mode. However, netcat lacks the --lua-exec option, which serves the purpose of executing an external Lua program against network sockets. This option is very handy for scripting tasks aimed at testing or debugging a wide range of services. The main strength of using this execution mode is that the programs are cross-platform as they are executed on the same built-in interpreter.

The httpd.lua script is an example distributed with Ncat to illustrate service emulation, but it should be clear that our options are endless. Lua is a very powerful language, and many tasks can be scripted with a few lines.

There's more...

Ncat offers a wide range of options that are documented thoroughly at https://nmap.org/ncat/guide/index.html. Do not forget to stop there and go over the full documentation.

Other ways of executing external commands with Ncat

Ncat supports three options to execute external programs:

--exec: This runs command without shell interpretation
--sh-exec: This runs command by passing a string to a system shell
--lua-exec: This runs Lua script using the built-in interpreter

Discovering systems with weak passwords with Ncrack

Ncrack is a network authentication cracking tool designed to identify systems with weak credentials. It is highly flexible and supports popular network protocols, such as FTP, SSH, Telnet, HTTP(S), POP3(S), SMB, RDP, VNC, SIP, Redis, PostgreSQL, and MySQL.

In this recipe, you will learn how to install Ncrack to find systems with weak passwords.

Getting ready

Grab the latest stable version of Ncrack from https://nmap.org/ncrack/. At the moment, the latest version is 0.5:

$wget https://nmap.org/ncrack/dist/ncrack-0.5.tar.gz

Untar the compressed file and enter the new directory:

$ tar -zxf ncrack-0.5.tar.gz 
$ cd ncrack-0.5

Configure and build Ncrack with the command:

$./configure && make

Finally, install it in your system:

#make install

Now you should be able to use Ncrack anywhere in your system.

How to do it...

To start a basic dictionary attack against a SSH server, use the following command:

$ncrack ssh://<target>:<port>

Ncrack will use the default settings to attack the SSH server running on the specified IP address and port. This might take some time depending on the network conditions:

   Starting Ncrack 0.5 ( http://ncrack.org ) at 2016-04-03 21:10 EEST  
   Discovered credentials for ssh on 192.168.1.2 22/tcp: 
   192.168.1.2 22/tcp ssh: guest 12345  
   Ncrack done: 1 service scanned in 56 seconds.  
   Ncrack finished.

In this case, we have successfully found the credentials of the account guest. Someone should have known better that 12345 is not a good password.

How it works...

Ncrack takes as arguments the hostname or IP address of the target and a service to attack. Targets and services can be defined as follows:

<[service-name]>://<target>:<[port-number]>

The simplest command requires a target and the service specification. Another way of running the scan shown earlier is as follows:

$ncrack 192.168.1.2:22
   Starting Ncrack 0.5 ( http://ncrack.org ) at 2016-01-03 22:10 EEST  
   Discovered credentials for ssh on 192.168.1.2 22/tcp: 
   192.168.1.2 22/tcp ssh: guest 12345 
   192.168.1.2 22/tcp ssh: admin money$  
   Ncrack done: 1 service scanned in 156.03 seconds.  
   Ncrack finished.

In this case, Ncrack automatically detected the SSH service based on the port number given in the target and performed a password auditing attack using the default dictionaries shipped with Ncrack. Luckily, this time we found two accounts with weak passwords.

There's more...

As we have seen Ncrack provides a few different ways of specifying targets, but it takes it to the next level with some interesting features, such as the ability to of pause and resume attacks. We will briefly explore some of its options, but I highly recommend you read the official documentation at https://nmap.org/ncrack/man.html for the full list of options.

Configuring authentication options

Ncrack would not be a good network login cracker without options to tune the authentication process. Ncrack users may use their own username and password lists with the options -U and -P correspondingly if the included lists (inside the directory /lists) are not adequate:

$ ncrack -U <user list file> -P <password list file> <[service-name]>://<target>:<[port-number]>

Otherwise, we might have a specific username or password we would like to test with the options --user and --pass:

$ ncrack --user <username> <[service-name]>://<target>:<[port-number]>
$ ncrack --pass <password> <[service-name]>://<target>:<[port-number]>

Pausing and resuming attacks

Ncrack supports resuming incomplete scans with the --resume option. If you had to stop a cracking session, just resume it passing the filename of the previous session:

$ncrack --resume cracking-session <[service-name]>://<target>:<[port-number]>

If we would like to set the filename of the session, use the --save option:

$ncrack --save cracking-session <[service-name]>://<target>:<[port-number]>

Launching Nmap scans remotely from a web browser using Rainmap Lite

Rainmap Lite is a web application designed for running Nmap scans from any web browser. It was designed to be light and to depend on as few dependencies as possible. It is perfect for installing on a remote server and then just logging in from your phone and scheduling scans when you are on the road.

In this recipe, you will learn how to launch a Nmap scan using Rainmap Lite.

Getting ready

To run Rainmap Lite, we need to download the code and run the application as follows:

Grab the latest stable version of Rainmap Lite:

$git clone https://github.com/cldrn/rainmap-lite.git

Install Django and the only project dependency, lxml:

$ pip install Django
$ pip install lxml

Change your working directory to the newly created folder and create the database schema:

$python manage.py migrate

Load the default scanning profiles:

$python manage.py loaddata nmapprofiles

Locate nmaper-cronjob.py and update the BASE_URL, SMTP_SERVER, SMTP_USER, SMTP_PASS, and SMTP_PORT variables to reflect your installation.
Run the application:

#python manage.py runserver 127.0.0.1:8080

Add a cron task that executes the agent periodically:

*/5 * * * * cd <App path> && /usr/bin/python nmaper-cronjob.py >> /var/log/nmaper.log 2>&1

And finally, don't forget to add an administrative user:

$ python manage.py createsuperuser

How to do it...

Point your favorite web browser to the URL where Rainmap Lite is running. If you follow the steps described previously, it should be running on port 8080.

The interface was designed to require as little typing as possible. Just fill in the field for target, select a scan profile from the drop-down list, and enter the e-mail address where you would like to receive the report. Hit SCAN when you are ready to add your scan to the queue:

How it works...

Rainmap Lite is a simple Django application that allows users to schedule and run Nmap scans from any web browser. The application was designed to be easy to install on any server, and it is great for installing on a remote VPS and use the interface to schedule scans and share the results with your team.

An important aspect is that it is based on a standard cron agent to reduce the number of dependencies. A more robust queue will probably be implemented in the future.

This project is very young and started as a personal project that I decided to share at Blackhat US Arsenal 2016. Feel free to send any bug report or suggestion to the project's GitHub page directly:

https://github.com/cldrn/rainmap-lite

There's more...

Scan profiles can be customized from the management console. The scanning profiles are updated in every version, and you are invited to contribute your own to the project's wiki at https://github.com/cldrn/rainmap-lite/wiki/Scanning-profiles.

Custom arguments

Custom arguments may be added on the fly without accessing the administration console by checking the box with the Custom Nmap arguments option: