As new functionalities were added to Nmap, new tools were written and incorporated into the Nmap family as sub-projects, such as Ndiff, Ncat, Ncrack, Zenmap, Nping, and even the Nmap Scripting Engine itself, to complement and extend the coverage of network-related tasks. These sub-projects were introduced throughout the years of the Nmap project participating in the Google Summer of Code program and they have become invaluable for the community as they serve specific needs.

This chapter will serve as an introduction to those who are unfamiliar with all the tools from the Nmap family and it will also show practical usage examples to those who know them but don't really use them. In this chapter, we will cover the following recipes:

• Monitoring servers remotely with Nmap and Ndiff
• Crafting ICMP echo replies with Nping
• Managing multiple scanning profiles with Zenmap
• Running Lua scripts against a network connection with Ncat
• Discovering systems with weak passwords with Ncrack
• Using Ncat to diagnose a network client
• Defending against Nmap service detection scans

By using tools from the Nmap project, we can set up a simple but effective monitoring system. Because our monitoring system will depend on Nmap, we can monitor not only open ports but any information the Nmap Scripting Engine can gather. To detect changes on the network, we will need to compare the results of two scans: the base or known good state and the current scan result. Consider the ports that you know that must be open as your base state.

Ndiff was designed to address the issues of using the traditional diff command with two Nmap scan results in XML format. It compares the files by removing false positives introduced by dynamic fields such as timestamps and producing a more human-friendly output.

This recipe describes how to use Bash scripting, cron, Nmap, and Ndiff to set up a monitoring system that alerts by email if changes are detected in a network or host.

Getting ready

In this recipe, we assume the system has been configured to send email via the mail command. If you would like to change the notification method, you simply need to update the Bash script and replace the mail command according to your preferred notification method. For example, you could use curl to make an HTTP request to your favorite social network or run a script that restarts the service.

How to do it...

To set up a simple monitoring system with Nmap, we are going to need to do a few things:

1. Create the /usr/local/share/nmap-mon/ directory (or whatever location you prefer) to store all the files required for our monitoring system.
2. Scan your targets and save the result in XML format in the directory that you just created:

   # nmap -oX base_results.xml -sV -n <target>

   The resulting base_results.xml file will be used as your base file, meaning that it should reflect the known good state of your network or host.

3. Create the nmap-mon.sh file in the directory you created earlier and paste the following code:

   #!/bin/bash
   #Bash script to email admin when changes are detected in a network using Nmap and Ndiff.
   #
   #Don't forget to adjust the CONFIGURATION variables.
   #Paulino Calderon <calderon@websec.mx>
   #
   #CONFIGURATION
   # 
   NETWORK="YOURTARGET" 
   ADMIN=YOUR@EMAIL.COM
   NMAP_FLAGS="-n -sV"
   BASE_PATH=/usr/local/share/nmap-mon/ 
   BIN_PATH=/usr/local/bin/ 
   BASE_FILE=base.xml 
   NDIFF_FILE=ndiff.log 
   NEW_RESULTS_FILE=newscanresults.xml 
   BASE_RESULTS="$BASE_PATH$BASE_FILE"
   NEW_RESULTS="$BASE_PATH$NEW_RESULTS_FILE" 
   NDIFF_RESULTS="$BASE_PATH$NDIFF_FILE"
   if [ -f $BASE_RESULTS ] then
     echo "Checking host $NETWORK"
     ${BIN_PATH}nmap -oX $NEW_RESULTS $NMAP_FLAGS $NETWORK
     ${BIN_PATH}ndiff $BASE_RESULTS $NEW_RESULTS > $NDIFF_RESULTS
     if [ $(cat $NDIFF_RESULTS | wc -l) -gt 0 ] then
       echo "Network changes detected in $NETWORK" 
       cat $NDIFF_RESULTS
       echo "Alerting admin $ADMIN"
       mail -s "Network changes detected in $NETWORK" $ADMIN < $NDIFF_RESULTS
     fi
   fi

4. Update the configuration values in the previous Bash script according to your system and needs:

   NETWORK="YOURTARGET" 
   ADMIN=YOUR@EMAIL.COM 
   NMAP_FLAGS="-sV -n -p-"
   BASE_PATH=/usr/local/share/nmap-mon/ 
   BIN_PATH=/usr/local/bin/ 
   BASE_FILE=base.xml 
   NDIFF_FILE=ndiff.log 
   NEW_RESULTS_FILE=newscanresults.xml

5. Make nmap-mon.sh executable by entering the following command:

   # chmod +x /usr/local/share/nmap-mon/nmap-mon.sh

6. Now run the nmap-mon.sh script to make sure it is working correctly:

   # /usr/local/share/nmap-mon/nmap-mon.sh

7. Launch your crontab editor to automatically execute the script periodically:

   # crontab -e

8. Add the following command:

   0 * * * * /usr/local/share/nmap-mon/nmap-mon.sh

You should now receive email alerts when Ndiff detects a change in your network.

How it works...

Ndiff is a tool for comparing two Nmap scans. Think about the traditional diff but for Nmap scan reports. With some help from Bash and cron, we set up a task that is executed at regular intervals to scan our network and compare our current state with an older state, to identify the differences between them. We used some basic Bash scripting to execute our monitoring scan and then executed Ndiff to compare the results:

if [ $(cat $NDIFF_RESULTS | wc -l) -gt 0 ] then
  echo "Network changes detected in $NETWORK"
  cat $NDIFF_RESULTS
  echo "Alerting admin $ADMIN"
  mail -s "Network changes detected in $NETWORK" $ADMIN < $NDIFF_RESULTS 
fi

There's more...

You can adjust the interval between scans by modifying the cron line:

0 * * * * /usr/local/share/nmap-mon/nmap-mon.sh

To update your base file, you simply need to overwrite your base file located at /usr/local/share/nmap-mon/. Remember that when we change the scan parameters to create our base file, we need to update them in nmap-mon.sh too.

Monitoring specific services

To monitor specific services, you need to update the scan parameters in nmap-mon.sh:

NMAP_FLAGS="-sV -Pn"

For example, if you would like to monitor a web server, you may use the following command:

NMAP_FLAGS="-sV --script http-google-safe -Pn -p80,443"

These options limit port scanning to ports 80 and 443; run the http-google-safe script to check whether your web server has been marked as malicious by the Google Safe Browsing service.

Nping is a utility designed to ease the process of crafting network packets. It is very useful to debug and troubleshoot network communications and perform traffic analysis.

This recipe will introduce Nping and go over the process of crafting and transmitting custom ICMP packets.

How to do it...

Let's say that we want to respond to an ICMP echo request packet with an echo reply using Nping. Consider that the first ICMP echo request packet has a source IP of 192.168.0.10 with an ICMP ID of 520, and the data string was the word ping. With that information, we can craft the reply with the following command:

#nping --icmp -c 1 --icmp-type 0 --icmp-code 0 --source-ip 192.168.0.5 --dest-ip 192.168.0.10 --icmp-id 520 --icmp-seq 0 --data-string 'ping'

In the output, you should see the sent ICMP echo reply packet with the values taken from the ICMP echo request packets:

SENT (0.0060s) ICMP [192.168.0.5 > 192.168.0.10 Echo reply (type=0/code=0) id=520 seq=0] IP [ttl=64 id=10898 iplen=32 ] 
Max rtt: N/A | Min rtt: N/A | Avg rtt: N/A
Raw packets sent: 1 (32B) | Rcvd: 0 (0B) | Lost: 1 (100.00%) Nping done: 1 IP address pinged in 1.01 seconds

How it works...

Nping allows configuring the values of most fields in TCP, UDP, ARP, and ICMP packets easily. The following command will send an ICMP echo reply packet with the values obtained from the ICMP echo request packet:

#nping --icmp -c 1 --icmp-type 0 --icmp-code 0 --source-ip 192.168.0.5 --dest-ip 192.168.0.10 --icmp-id 520 --icmp-seq 0 --data-string 'ping'

Let's break it down by its arguments:

• --icmp: Sets ICMP as the protocol to use.
• -c 1: Packet count. Sends only one packet.
• --icmp-type 0 --icmp-code 0: Sets the ICMP type and code. This type corresponds to an echo reply message.
• --source-ip 192.168.0.5 --dest-ip 192.168.0.10: Sets the source and destination IP address.
• --icmp-id 520: Sets the ICMP identifier of the request packet.
• --icmp-seq 0: Sets the ICMP sequence number.
• --data-string 'ping': Sets the data string.

There's more...

Nping can set most fields in TCP, UDP, ARP, and ICMP packets via arguments but offers a lot more customization than we offer. In addition to the interesting timing and performance options, Nping supports a mode named echo that is handy when troubleshooting firewall or routing issues. I highly recommend you go over the documentation at https://nmap.org/nping/ to become familiar with this powerful tool and more scenarios where it can be handy.

Scanning profiles are a combination of Nmap options that can be used to save time when launching Nmap scans.

This recipe is about adding, editing, and deleting a scanning profile in Zenmap.

How to do it...

Let's add a new profile for scanning web servers:

1. Launch Zenmap.
2. Click on Profile on the main toolbar.
3. Click on New Profile or press Ctrl + P. Profile Editor will be launched.
4. Enter a profile name and a description on the Profile tab.
5. Enable Version detection and select TCP connect scan (-sT) in the Scan tab.
6. Enable Don't ping before scanning (-Pn) in the Ping tab.
7. Enable the following scripts on the Scripting tab:

   http-backup-finder

   http-config-backup

   http-cors

   http-cross-domain-policy

   http-csrf

   http-dombased-xss

   http-enum

   http-favicon

   http-headers

   http-methods

   http-open-redirect

   http-robots.txt

   http-server-header

   http-svn-info

   http-title

8. Next, go to the Target tab and click on Ports to scan (-p) and enter 80, 443.
9. Save your changes by clicking on Save Changes:

Figure 2.1 – NSE script selection in Zenmap

Your new scanning profile should be available from the Profile drop-down menu. We selected some of the available scripts to give you an idea, but you can adjust the scan according to your needs.

How it works...

After using the editor to create our profile, we are left with the following Nmap command:

$ nmap -sT -sV -p 80,443 -T4 -v -Pn --script http-backup-finder,http-config-backup,http-cors,http-cross-domain-policy,http-csrf,http-dombased-xss,http-enum,http-headers,http-methods,http-open-redirect,http-robots.txt,http-server-header,http-title <target>

Using the Profile wizard, we have enabled service scanning (-sV), set the scanning ports to 80 and 443, disabled host discovery (-Pn), and selected a bunch of HTTP-related scripts to gather as much information as possible from this web server. We now have this command saved and easily accessible for our scanning activities against new targets in the future.

There's more...

Customizing scan profiles can be done through the user interface. Default scanning profiles can be used as templates when creating new ones. Let's review how we work with the scanning profiles.

Zenmap scanning profiles

The predefined Zenmap scanning profiles help newcomers familiarize themselves with Nmap. I recommend that you analyze them to understand the scanning techniques available in Nmap along with some useful combinations of its options:

• Intense scan: nmap -T4 -A -v
• Intense scan plus UDP: nmap -sS -sU -T4 -A -v
• Intense scan, all TCP ports: nmap -p 1-65535 -T4 -A -v
• Intense scan, no ping: nmap -T4 -A -v -Pn
• Ping scan: nmap -sn
• Quick scan: nmap -T4 -F
• Quick scan plus: nmap -sV -T4 -O -F --version-light
• Quick traceroute: nmap -sn --traceroute
• Regular scan: nmap
• Slow comprehensive scan: nmap -sS -sU -T4 -A -v -PE -PP -PS80,443 - PA3389 -PU40125 -PY -g 53 --script "default or discovery and safe"

  Important note

  You can find other scanning profiles in the database of Rainmap Lite at https://github.com/cldrn/rainmap-lite/wiki/Scanning-profiles.

Editing or deleting a scan profile

To edit or delete a scan profile, you need to select the entry you wish to modify from the Profile drop-down menu. Click on Profile on the main toolbar and select Edit Selected Profile (Ctrl + E).

The editor will be launched, allowing you to edit or delete the selected profile.

Ncat allows users to read, write, redirect, and modify network data in some very interesting ways. Think about it as an enhanced version of the traditional tool netcat. Ncat offers the possibility of running external commands once a connection has been established successfully. Users may use Lua scripts to perform actions on the network sockets created by Ncat.

The following recipe will show you how to run an HTTP server contained in a Lua script with Ncat.

How to do it...

1. Running Lua scripts against network connections in Ncat is very straightforward; just use the --lua-exec option to set the Lua script you want to execute and the listening port or host to connect to:

   $ncat --lua-exec <path to Lua script> --listen <port>

2. To start a web server with Ncat, locate the httpd.lua script inside your Ncat's script folder and use the following command:

   $ncat --lua-exec /path/to/httpd.lua --listen 8080 --keep-open

3. Ncat will start listening on port 8080 and execute the specified Lua script. You may verify that the script is running correctly by pointing a web browser in that direction and checking whether the Got a request for message appears on the output.

How it works...

If you have used netcat before, you are already halfway there. Similar to netcat, Ncat can be put into listening (--listen) and connect mode. However, netcat lacks the --lua-exec option, which serves the purpose of executing an external Lua script against the network socket. This option is very handy for scripting tasks aimed at testing or debugging a wide range of services. The main strength of using this execution mode is that the programs are cross-platform as they are executed on the same built-in interpreter.

The httpd.lua script is an example distributed with Ncat to illustrate service emulation, but it should be clear that our options are endless. Lua is a very powerful language, and many tasks can be scripted with very few lines.

There's more...

Ncat offers a wide range of options that are documented thoroughly at https://nmap.org/ncat/guide/index.html. Do not forget to stop there and go over the full documentation.

Other ways of executing external commands with Ncat

Ncat supports three options to execute external programs:

• --exec: This runs commands without shell interpretation.
• --sh-exec: This runs commands by passing a string to a system shell.
• --lua-exec: This runs a Lua script using the built-in interpreter.

Ncrack is a network authentication cracking tool designed to identify systems with weak credentials. It is highly flexible and supports popular network protocols, such as FTP, SSH, Telnet, HTTP(S), POP3(S), SMB, RDP, VNC, SIP, Redis, PostgreSQL, and MySQL.

In this recipe, you will learn how to install Ncrack to find systems with weak passwords.

Getting ready

Grab the latest stable version of Ncrack from https://nmap.org/ncrack/. At the moment, the latest version is 0.7:

$wget https://nmap.org/ncrack/dist/ncrack-0.7.tar.gz

Decompress the file and enter the new directory:

$ tar -zxf ncrack-0.7.tar.gz
$ cd ncrack-0.7

Configure and build Ncrack with the following command:

$./configure && make

Finally, install it in your system:

#make install

Now you should be able to use Ncrack anywhere in your system.

How to do it...

To start a basic dictionary attack against an SSH server, use the following command:

$ncrack ssh://<target>:<port>

Ncrack will use the default settings to attack the SSH server running on the specified IP address and port. This might take some time depending on the network conditions:

Discovered credentials for ssh on 192.168.1.2 22/tcp:
192.168.1.2 22/tcp ssh: guest 12345
Ncrack done: 1 service scanned in 56 seconds. Ncrack finished.

In this case, we have successfully found the credentials of the account guest. Someone should have known that 12345 is not a good password.

How it works...

Ncrack takes as arguments the hostname or IP address of the target and a service to attack. Targets and services can be defined as follows:

<[service-name]>://<target>:<[port-number]>

The simplest command requires a target and the service specification. Another way of running the scan shown earlier is as follows:

$ncrack 192.168.1.2:22
Starting Ncrack 0.7 ( http://ncrack.org ) at 2020-10-08 22:10 EST Discovered credentials for ssh on 192.168.1.2 22/tcp:
192.168.1.2 22/tcp ssh: guest 12345 192.168.1.2 22/tcp ssh: admin money$
Ncrack done: 1 service scanned in 156.03 seconds. Ncrack finished.

In this case, Ncrack automatically detected the SSH service based on the port number given in the target and performed a password auditing attack using the default dictionaries shipped with Ncrack. Luckily, this time we found two accounts with weak passwords.

There's more...

As we have seen, Ncrack provides a few different ways of specifying targets, but it takes it to the next level with some interesting features, such as the ability to pause and resume attacks. We will briefly explore some of its options, but I highly recommend you read the official documentation at https://nmap.org/ncrack/man.html for the full list of options.

Configuring authentication options

Ncrack would not be a good network login cracker without options to tune the authentication process. Ncrack users may use their own username and password lists with the -U and -P options correspondingly if the included lists (inside the /lists directory) are not adequate:

$ ncrack -U <user list file> -P <password list file> <[service- name]>://<target>:<[port-number]>

Otherwise, we might have a specific username or password we would like to test with the --user and --pass options:

$ ncrack --user <username> <[service-name]>://<target>:<[port-number]>
$ ncrack --pass <password> <[service-name]>://<target>:<[port-number]>

Pausing and resuming attacks

Ncrack supports resuming incomplete scans with the --resume option. If you had to stop a cracking session, just resume it by passing the filename of the previous session:

$ncrack --resume cracking-session <[service-name]>://<target>:<[port-number]>

If you would like to set the filename of the session to resume it later in case you need to, use the --save option:

$ncrack --save cracking-session <[service-name]>://<target>:<[port-number]>

Ncat can be used for a wide range of tasks including diagnosing network communications. The ability to easily set it up as a proxy is helpful when we need to analyze the traffic sent by a network client. With the help of Ncat, we can analyze the data exchanged and identify possible errors.

This recipe describes how to use Ncat to analyze network communications between a remote server and our local client.

How to do it...

Start a local listener with Ncat:

$ncat -l -k 5555 --hex-dump client.txt

We now have a listener on localhost port 5555. It is time to configure our client to connect to our local IP address (it works on remote IP addresses as well). Connect to our listener to see the traffic that is sent by the client. For example, to see what probes are sent during a service scan, we use this:

$nmap -sV -p 5555 localhost

The traffic sent will be displayed as the output of our first ncat command:

$ncat -l -k 5555 --hex-dump client.txt
versionbind??SMB@@?PC NETWORK PROGRAM 1.0MICROSOFT NETWORKS 1.03MICROSOFT NETWORKS 3.0LANMAN1.0LM1.2X002SambaNT LANMAN 1.0NT LM 0.12CNXN2????host::GET / HTTP/1.0
OPTIONS / HTTP/1.0
OPTIONS / RTSP/1.0
?(r????|

Depending on the client, a configuration might support proxies out of the box. If not, use the target IP address to the host where your listener is running. Note that you may not be able to change the port, but you can use the same port on your local machine to work around this. The hex dump will be saved in the client.txt file:

Figure 2.2 – Hex dump of traffic sent by the client

How it works...

The ncat command starts a listener on localhost port 5555 (-l 5555) that accepts multiple connections (-k) and dumps the output in hexadecimal format (--hex-dump client.txt). In this case, Ncat acts as a proxy between the local or remote server and our client (Nmap) and the client is instructed to connect to the proxy. Note that in this example we are not re-routing the network traffic, but it is possible. The output shown by Ncat is the traffic sent by the client.

The interesting option here is --hex-dump, which allows us to see those unprintable characters usually found in network traffic. Hex format makes it easier to analyze and compare with the expected results. If something is not being sent correctly, we would catch it here after reading the output.

There is more...

Since Ncat supports encrypted channels out of the box, a simple solution to upgrade services that use plain text to communicate is tunneling the traffic in an encrypted channel with Ncat. Ncat can chain multiple commands to achieve this – as here, for example:

ncat -l localhost 143 --sh-exec "ncat --ssl imap.packtpub.com 993"

Once the client connects to local port 143, it connects to imap.packtpub.com using an encrypted channel (--ssl). When the network traffic leaves the box, it will be using the SSL channel.

If you belong to the blue team of an organization, it is likely you are already running a decoy host in your network. But what about something that slows down attackers? As Nmap is one of the most popular tools for port scanning, it is a good idea to implement something that will hinder the scans.

In this recipe, you will learn how to make Nmap scan indefinitely when a service detection scan is used against a target.

How to do it...

To start a fake HTTP service that sends random data indefinitely on a Linux-based host, use the following Ncat command:

$ncat -l 127.0.0.1 8080 -c "echo 'HTTP/1.1 200 OK\r\n\r\n'; cat /dev/urandom" -k

A new service running on port 8080 will start on your localhost. If an attacker uses Nmap's service detection scan (-sV), the service will prevent Nmap from closing the network socket, and hence the scan will never finish.

How it works...

The previous Ncat command simply listens on the local IP address TCP port 8080 (-l 127.0.0.1 8080) and executes a system command using the -c option. The -k option is also used to enable multiple connections so the socket is not closed after the first client connects. The executed system command is composed of two parts:

• A fake protocol header: echo 'HTTP/1.1 200 OK\r\n\r\n'
• Random data stream: cat /dev/urandom

The fake application protocol header is used to confuse the scanner making it launch a read operation that will only close once data transmission is complete, which will never happen. Additionally, Nmap prints results only when all hosts are processed, and if one host isn't complete, none of the results from that group are printed, so the attackers won't be able to see the incomplete results report. By using the /dev/urandom pseudo-device, we generate an infinite body message to append to the response and achieve this infinite response condition.

There's more...

Even though it is basic, this technique is pretty effective and does not only work on Nmap. You would be surprised how fragile some vulnerability scanners are, and this is only one method for hindering their results. You should get creative and analyze how the scanner works to identify possible attack vectors. In this recipe, we used an HTTP header to trick the scanner, but other protocols could also be susceptible.

Attacking web crawlers in security scanners

Writing web crawlers and handling the infinite combination of tags and fields in poorly written HTML is difficult. Scanners often include web crawlers to enumerate the attack surface and even detect vulnerabilities. By targeting the web crawler engine in scanners, we may affect the scanning behavior. Common attacks against web crawlers include web servers with link loops, pages with a high number of nested links, and dynamic content generation, among many others. Try these techniques against security scanners to discover interesting defense techniques!