Mastering the Nmap Scripting Engine

By Paulino Calderon
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Introduction to the Nmap Scripting Engine

About this book

Nmap is a well-known security tool used by penetration testers and system administrators for many different networking tasks. The Nmap Scripting Engine (NSE) was introduced during Google's Summer of Code 2006 and has added the ability to perform additional tasks on target hosts, such as advanced fingerprinting and service discovery and information gathering.

This book will teach you everything you need to know to master the art of developing NSE scripts. The book starts by covering the fundamental concepts of Lua programming and reviews the syntax and structure of NSE scripts. After that, it covers the most important features of NSE. It jumps right into coding practical scripts and explains how to use the Nmap API and the available NSE libraries to produce robust scripts. Finally, the book covers output formatting, string handling, network I/O, parallelism, and vulnerability exploitation.

Publication date:
February 2015
Publisher
Packt
Pages
244
ISBN
9781782168317

 

Chapter 1. Introduction to the Nmap Scripting Engine

The Nmap Scripting Engine (NSE) revolutionized the capabilities of Nmap. It was introduced during Google's Summer of Code in 2007, and it has become an arsenal by itself with almost 500 official scripts. Although the first scripts were aimed at improving service and host detection, people quickly started submitting scripts for other tasks. Today, there are 14 categories covering a broad range of tasks, from network discovery to detection and exploitation of security vulnerabilities. You can use NSE scripts to brute-force accounts with weak passwords, find online hosts with different broadcast requests, sniff the network, discover forgotten backup files in web servers, detect the latest SSL 3.0 vulnerability known as Poodle, and even exploit vulnerabilities in popular software.

The script collection grows rapidly, so I recommend staying up-to-date by subscribing to the Nmap Development mailing list, located at http://nmap.org/mailman/listinfo/dev. Nmap's community is very active, so I encourage you to always keep an updated copy among your penetration testing tools.

NSE scripts are great for drafting proof-of-concept code since the modules are written in Lua, a simple yet powerful language. It allows us to quickly program any task we have in mind with the help of the available NSE libraries. Its flexible syntax is easy to learn, and I'm sure you will find yourself loving it after experimenting with it for a day.

This chapter will introduce you to NSE, covering several topics from installation and development environment setup to advanced usage tips. If you are familiar with the following topics, you may skip this chapter:

  • Building Nmap from source code

  • Running NSE scripts

  • Passing arguments to NSE scripts

  • Scanning phases

  • NSE applications

  • Setting up a development environment

If you are not familiar with NSE already, this chapter will get you prepared for what is coming in the next chapters. For those with some experience, I still recommend going through this chapter as I'm including advanced tips related to script selection and usage. Fire up your terminals and let's get to work.

 

Installing Nmap


Nmap binaries for all major platforms can be found at the official website, at http://nmap.org/download.html. A lot of distributions also offer official packages. However, if you want the latest features and NSE scripts, you need to work with the development branch. The code in this branch is more stable than the name implies, as the developers make sure their code is working before submitting it to this branch. By always running a copy from the development branch, you also always have the latest bug fixes.

Building Nmap from source code

Nmap uses Subversion, the famous Version Control System (VCS), to manage the source code of the project. First, make sure you have a Subversion client at hand:

$svn --version

On Debian-based systems, you can install Subversion by running the following command:

#apt-get install subversion

Note

A good alternative to Subversion is RapidSVN, a cross-platform Subversion client with a Graphical User Interface. You can get RapidSVN from http://rapidsvn.tigris.org/.

Once the Subversion client is installed, we grab the development branch from the official repositories with the following command:

$svn co https://svn.nmap.org/nmap

The preceding command downloads the latest revision of the development branch into a new directory in your current directory. We will refer to this folder as your working copy. Before compiling, you may need additional tools and libraries such as OpenSSL. Make sure you have all the requirements installed by running the following command:

#apt-get install make g++ libssl-dev autoconf

Now we can compile and install Nmap. Go to the nmap directory that was just created by Subversion and enter the following command:

$./configure

If everything worked correctly, you should see an ASCII dragon warning you about the power of Nmap, like this:

Now let's compile Nmap with the following commands:

$make
#make install

Tip

In BSD systems, run gmake instead of make.

Now run Nmap to ensure that it was installed correctly. You should see your build information:

#nmap -v
Nmap version 6.47SVN ( http://nmap.org )
Platform: x86_64-apple-darwin14.0.0
Compiled with: nmap-liblua-5.2.3 openssl-0.9.8za nmap-libpcre-7.6 libpcap-1.5.3 nmap-libdnet-1.12 ipv6
Compiled without:
Available nsock engines: kqueue poll select

Keeping Nmap up to date

To update your working copy, use the following command inside your working directory:

$svn up

Once your working copy is synchronized with the remote repository, we need to rebuild Nmap:

$make

Note

In BSD systems, run gmake.

Again to install the binaries and data files in the system, use this command:

#make install
 

Running NSE scripts


NSE was designed with flexibility in mind, and supports several features to control the execution of NSE scripts. In this chapter, we will learn not only that NSE scripts can be executed during different scan phases but also that they can be selected with a high level of granularity, depending on host conditions. In combination with robust libraries and plenty of configuration options, NSE offers a level of flexibility that is hard to match in other tools and frameworks.

We can begin testing NSE against the host, scanme.nmap.org. This server is managed by the Nmap project and allows users to scan it as long as the scans are not too intrusive. Let's begin by running a scan with version detection and NSE enabled against our test target—scanme.nmap.org:

#nmap –sV –sC -O scanme.nmap.org

You should see something similar to this:

The previous command ran a SYN scan with OS detection (-O), service detection (-sV), and most importantly with NSE on (-sC). The -sC option enables the NSE and runs any script in the default category. This set of scripts is considered safe as it won't perform any operations that could interfere with a service running on the target host. However, note that some of the scripts perform actions that can raise alarms in intrusion detection systems (IDS) and intrusion prevention systems (IPS).

Note

An unprivileged scan can't access raw sockets, which generally results in a slower scan. However, the SYN scan is the default type of scan executed when Nmap runs in privileged mode.

The safe category contains scripts such as these:

  • banner: This prints the response of a TCP connection to an open port

  • broadcast-ping: This discovers hosts with broadcast pings

  • dns-recursion: This detects DNS servers that allow recursion that may be used in DNS amplification attacks

  • upnp-info: This extracts information from the upnp service

  • firewalk: This attempts to discover firewalls using an IP TTL expiration technique

The previously mentioned scripts are only a few compared to the current total of almost 500. That's a whole lot more of information that can be collected by simply using NSE.

 

Script categories


The collection of NSE scripts is divided into the following categories:

Script category

Description

auth

NSE scripts related to user authentication.

broadcast

A very interesting category of scripts that use broadcast petitions to gather network information.

brute

A category for scripts that help conduct brute-force password auditing.

default

Scripts executed when a script scan is executed (-sC).

discovery

Scripts related to host and service discovery.

dos

Scripts related to denial-of-service attacks.

exploit

Scripts used to exploit security vulnerabilities.

external

This category is for scripts depending on a third-party service.

fuzzer

NSE scripts focused on fuzzing.

intrusive

A category for scripts that might crash something or generate a lot of network noise. Scripts that system administrators may consider intrusive go here.

malware

A category for scripts related to malware detection.

safe

Scripts that are considered safe in all situations.

version

Scripts for advanced version detection.

vuln

Scripts related to detecting and exploiting security vulnerabilities.

NSE script selection

Nmap supports the --script option for script selection. This option can take a script name, NSE category, a path to a NSE file, a folder containing scripts, or even an expression. Expressions allow incredible flexibility when selecting scripts, as we will see in the following sections.

Selecting by script name or category

You can execute scripts by their name using the --script Nmap option. Execute several scripts at once by separating them with a comma:

nmap --script http-title <target>
nmap -p80 --script http-huawei-hg5xx-vuln <target>
nmap --script http-title,http-methods <target>

The following screenshot shows the output of the http-huawei-hg5xx-vuln script. This script exploits a remote vulnerability in Huawei devices to retrieve sensitive information, which includes the PPPoE credentials and the wireless security configuration:

PORT   STATE SERVICE VERSION
80/tcp open  http    Huawei aDSL modem EchoLife HG530 (V100R001B122gTelmex) 4.07 -- UPnP/1.0 (ZyXEL ZyWALL 2)
| http-huawei-hg5xx-vuln:
|   VULNERABLE:
|   Remote credential and information disclosure in modems Huawei HG5XX
|     State: VULNERABLE (Exploitable)
|     Description:
|       Modems Huawei 530x, 520x and possibly others are vulnerable to remote credential and information disclosure.
|       Attackers can query the URIs "/Listadeparametros.html" and "/wanfun.js" to extract sensitive information
|       including PPPoE credentials, firmware version, model, gateway, dns servers and active connections among other values
|     Disclosure date: 2011-01-1
|     Extra information:
|
|   Model:EchoLife HG530
|   Firmware version:V100R001B122gTelmex
|   External IP:xxx.xxx.xx.xxx
|   Gateway IP:xxx.xx.xxx.xxx
|   DNS 1:200.33.146.249
|   DNS 2:200.33.146.241
|   Network segment:192.168.1.0
|   Active ethernet connections:0
|   Active wireless connections:3
|   BSSID:0xdeadbeefcafe
|   Wireless Encryption (Boolean):1
|   PPPoE username:xxx
|   PPPoE password:xxx
|     References:
|       http://routerpwn.com/#huawei
|_      http://websec.ca/advisories/view/Huawei-HG520c-3.10.18.x-information-disclosure

To select a whole category, simply use the name of the category (see the Script categories section) as the argument. For example, to run the exploit category, use the following command:

nmap --script exploit <target>

You can also run several categories by separating them with a comma:

nmap --script discovery,intrusive <target>

Note

The -sC option is merely an alias of the --script default option.

Selecting by filename or folder

To execute a NSE script file, use this command:

nmap --script /path/to/script.nse <target>

Similarly with categories, you can execute several scripts by separating the paths with a comma:

nmap --script /path/to/script.nse,/another/path/script2.nse <target>

To execute all the scripts contained in a folder, you only need to pass the folder name as an argument:

nmap --script /path/to/folder/ <target>
nmap --script /custom-nse-scripts/ scanme.nmap.org

Tip

Keep in mind that the --script option accepts relative and absolute paths to scripts and folders. Besides the current directory, relative paths can be looked for in the following directories:

  • --datadir

  • $NMAPDIR

  • ~/.nmap

  • %HOMEPATH%\AppData\Roaming\nmap

  • The directory containing nmap

  • The directory containing nmap followed by this relative path: ../share/nmap

  • NMAPDATADIR

Advanced script selection with expressions

Expressions are used to describe a set of scripts. Let's go through the different scenarios where we can take advantage of script selection with expressions:

  • For example, the not exploit expression will match any script that does not belong to the exploit category:

    #nmap -sV --script "not exploit" <target>
    
  • The or and and operators allow us to construct more complex expressions. The following expression will match any script that is not in the intrusive, dos, or exploit categories:

    #nmap --script "not(intrusive or dos or exploit)" -sV <target>
    
  • If we would like to execute all scripts in the broadcast and discovery categories, we use this:

    #nmap --script "broadcast and discovery" <<target>
    
  • If you are selecting scripts, you can also use the wildcard character, *:

    #nmap --script "snmp-*" <target>
    
  • Of course, we can combine wildcards and expressions. For example, let's run all the scripts whose names begin with http-, but exclude the http-slowloris, http-brute, http-form-fuzzer, and http-enum scripts:

    #nmap --script "http-* and not(http-slowloris or http-brute or http-enum or http-form-fuzzer)" <target>
    
  • We can also combine wildcard selection with expressions when selecting categories. The next command executes all scripts whose names begin with http- that are not listed in the exploit category:

    #nmap --script "http-* and not(exploit)" <target>
    

NSE script arguments

The --script-args Nmap option is used to set arguments in NSE scripts. For example, if you would like to set the http library argument, useragent, You can use this expression:

$nmap -sV --script http-title --script-args http.useragent="Mozilla 1337" <target>

Not a lot of Nmap users know this but you can also omit the script name when setting arguments:

$nmap -p80 --script http-trace --script-args path <target>

You can use the preceding expression instead of using this:

$nmap -p80 --script http-trace --script-args http-trace.path <target>

If you are working with scripts that share argument names, you must avoid name conflicts manually:

$nmap --script http-majordomo2-dir-traversal,http-axis2-dir-traversal --script-args http-axis2-dir-traversal.uri=/axis2/,uri=/majordomo/ <target> 
$nmap --script http-majordomo2-dir-traversal,http-axis2-dir-traversal --script-args uri=/axis2/,http-majordomo2-dir-traversal.uri=/majordomo/ <target> 
$nmap --script http-majordomo2-dir-traversal,http-axis2-dir-traversal --script-args http-axis2-dir-traversal.uri=/axis2/,http-majordomo2-dir-traversal.uri=/majordomo/ <target>

Note

The alias in script arguments will only work if the NSE script uses the stdnse.get_script_args()function to load the arguments (refer to Chapter 4, Exploring the Nmap Scripting Engine API and Libraries). You are encouraged to always use this function, but there are a few scripts that were submitted before the function was introduced.

Loading script arguments from a file

If you are planning to run several scans, it is probably a good idea to write down your script arguments in a file to save some typing. NSE supports loading NSE arguments from an absolute or relative path with the --script-args-file option. The arguments contained in the file must be separated by commas or new lines:

nmap --script "discovery,broadcast" --script-args-file nmap-args.txt <target>

The contents of the nmap-args.txt file are as follows:

http.useragent=Not Nmap
http.max-connections=50
userdb=/path/to/usernames.lst
passdb=/path/to/dictionary.lst

Forcing the execution of NSE scripts

Nmap can force the execution of a NSE script by prepending + to the script name:

$nmap --script +<script selection> <<arg1, arg2, …>

Let's say we want to force the execution of the http-title NSE script against the service running on port 1212:

$nmap --script +http-title -p1212 192.168.1.210

Without the + sign, the script will not run but, since we added it, the report comes back with the following:

Nmap scan report for 192.168.1.210
Host is up (0.00026s latency).
PORT     STATE SERVICE
1212/tcp open  lupa
|_http-title: W00t!

Debugging NSE scripts

If you need to analyze the traffic sent and received by NSE, use the --script-trace option. For example, if you would like to see the payloads sent by the NSE scripts in the exploit category, you can use this expression:

#nmap --script exploit --script-trace <target>

You can also turn on the debugging mode of Nmap with the -d[1-9] flag. This flag can be followed by an integer that denotes the debugging level and should be between 1 and 9. The higher the level, the more verbose is the output:

#nmap -sV –-script exploit -d3 <target> 

The --packet-trace option includes all the packets sent and received, not only the traffic generated by NSE:

#nmap -O --script myscript.nse --packet-trace <target>
 

Scan phases and NSE


Nmap scans are divided into several phases but NSE is only involved in three of them: pre-scanning, script scanning, and post-scanning. The execution rule defined by a function in the NSE script determines whether it runs in any of those phases.

Note

To learn more about the phases of Nmap scans, check out Appendix A, Scan Phases.

NSE script rules

NSE scripts can have one of four different types of execution rule:

  • prerule

  • postrule

  • portrule

  • hostrule

Let's review some examples of these different script rules. This will also help you learn to debug scripts for those times when you run into problems:

  • prerule(): The following is a snippet from the targets-sniffer.nse NSE script. It illustrates how we can use a prerule function to check whether Nmap is running in privileged mode and whether it can determine the network interface correctly:

    prerule = function()
      return nmap.is_privileged() and 
        (stdnse.get_script_args("targets-sniffer.iface") or nmap.get_interface())
  • postrule(): The ssh-hostkey script uses a postrule function to detect hosts that share the same SSH public keys:

    postrule = function() return (nmap.registry.sshhostkey ~= nil) end
  • portrule(host, port): The following is a snippet of the portrule function of the jdwp-inject script. This portrule function will match a service detection string and specific port protocol and state:

    portrule = function(host, port)
            -- JDWP will close the port if there is no valid handshake within 2
            -- seconds, Service detection's NULL probe detects it as tcpwrapped.
            return port.service == "tcpwrapped"
                   and port.protocol == "tcp" and port.state == "open"
                   and not(shortport.port_is_excluded(port.number,port.protocol))
    end
  • hostrule(): The sniffer-detect script's host rule determines that the script will only execute with local Ethernet hosts:

    hostrule = function(host)
            if nmap.address_family() ~= 'inet' then
                    stdnse.print_debug("%s is IPv4 compatible only.", SCRIPT_NAME)
                    return false
            end
            if host.directly_connected == true and
              host.mac_addr ~= nil and
              host.mac_addr_src ~= nil and
              host.interface ~= nil then
                    local iface = nmap.get_interface_info(host.interface)
                    if iface and iface.link == 'ethernet' then
                          return true
                    end
            end
            return false
    end
 

Applications of NSE scripts


As you probably know by now, the applications of NSE cover a wide range of tasks. Nmap gives access to NSE developers to a "host and port" table containing relevant information collected during the scan, such as service name, operating system, protocol, and so on. The information available depends on the options used during the scan.

Unfortunately, there is not enough space in one chapter to cover all the great NSE scripts. If you are interested in learning more applications, I recommend checking out my previous book named Nmap 6: Network Exploration and Security Auditing Cookbook, Paulino Calderón Pale, Packt Publishing, where I covered in detail over 120 different tasks that can be done with Nmap. Its official website is at http://nmap-cookbook.com.

Information-gathering

Information-gathering is one of the strengths of NSE, and the collection of scripts available is astonishing. These scripts use different techniques and data sources to obtain additional host information such as virtual hosts, service versions, user lists, and even geolocation. Keep in mind that some of these scripts query external services, and the accuracy of the information depends on each database.

Collecting UPNP information

UPNP protocols were designed to allow network devices to find each other, and some serious flaws have been discovered in a lot of implementations of these sets of protocols. The upnp-info script was designed to query a UPNP service to obtain additional information about the device:

#nmap -sU -p1900 --script upnp-info <target>

If the preceding command runs successfully, the amount of information returned by the service depends on the type of device and UPNP implementation:

Nmap scan report for 192.168.1.1
Host is up (0.067s latency).
PORT     STATE SERVICE
1900/udp open  upnp
| upnp-info: 
| 192.168.1.1
|     Server: Custom/1.0 UPnP/1.0 Proc/Ver
|     Location: http://192.168.1.1:5431/dyndev/uuid:3872c05b-c117-17c1-5bc0-12345
|       Webserver: LINUX/2.4 UPnP/1.0 BRCM400/1.0
|       Name: Broadcom ADSL Router
|       Manufacturer: Comtrend
|       Model Descr: (null)
|       Model Name: AR-5381u
|       Model Version: 1.0
|       Name: WANDevice.1
|       Manufacturer: Comtrend
|       Model Descr: (null)
|       Model Name: AR-5381u
|       Model Version: 1.0
|       Name: WanConnectionDevice.1
|       Manufacturer: Comtrend
|       Model Descr: (null)
|       Model Name: AR-5381u
|_      Model Version: 1.0

Finding all hostnames resolving to the same IP address

The hostmap-* set of scripts lists all the hostnames pointing to the same IP address. This is useful when working with web servers that return different content depending on the hostname header. Currently, there are three scripts:

  • hostmap-bfk

  • hostmap-robtex

  • hostmap-ip2hosts

We can run them at the same time with the following command:

$nmap -sn --script "hostmap*" <target>

If there are any records on the external databases, they will be shown in the results:

Nmap scan report for packtpub.com (83.166.169.228)
Host is up (0.13s latency).

Host script results:
| hostmap-bfk: 
|   hosts: 
|     packtpub.com
|_    83.166.169.228
| hostmap-robtex: 
|   hosts: 
|_    packtpub.com
| hostmap-ip2hosts: 
|   hosts: 
|     www.packtpub.com
|     packtpub.com
|_    83.166.169.228

Advanced host discovery

The flexibility of allowing pre-scanning and post-scanning scripts gives us the ability to include targets on-the-fly, analyze scan results, and even launch additional probes to detect more target hosts. The broadcast NSE category collects a very interesting set of scripts that doesn't send traffic directly to the target host using multicast requests. On the other hand, some scripts (such as targets-sniffer) merely listen to the local network to find new targets, without generating any traffic.

Discovering hosts with broadcast pings

The broadcast-ping script attempts to discover hosts by sending a ping request to the broadcast address, 255.255.255.255. The machines configured to respond to broadcast requests will reveal themselves:

# nmap --script broadcast-ping 
Pre-scan script results: 
| broadcast-ping: 
|   IP: 192.168.1.202  MAC: 08:00:27:16:4f:71 
|   IP: 192.168.1.206  MAC: 40:25:c2:3f:c7:24 
|_  Use --script-args=newtargets to add the results as targets 
WARNING: No targets were specified, so 0 hosts scanned. 
Nmap done: 0 IP addresses (0 hosts up) scanned in 3.25 seconds 

All the hosts that responded to the broadcast ping will be shown. Additionally, using the newtargets argument, these hosts will be added to the scan queue:

# nmap --script broadcast-ping --script-args newtargets
Starting Nmap 6.47SVN ( http://nmap.org ) at 2014-11-30 22:05 CST
Pre-scan script results:
| broadcast-ping: 
|_  IP: 192.168.0.8  MAC: 6c:ad:f8:7b:83:ab
Nmap scan report for 192.168.0.8
Host is up (0.0083s latency).
Not shown: 998 closed ports
PORT     STATE SERVICE
8008/tcp open  http
8009/tcp open  ajp13
MAC Address: 6C:AD:F8:7B:83:AB (Azurewave Technologies)

Listening to your LAN to discover targets

The targets-sniffer script is very peculiar because it is one of the few scripts that actually sniff a LAN network in order to discover new local targets. This script requires privileged mode and that you set the interface for use with the -e Nmap option:

#nmap -sL --script=targets-sniffer -e <interface>
Starting Nmap 6.47SVN ( http://nmap.org ) at 2014-11-30 22:11 CST
Pre-scan script results:
| targets-sniffer: Sniffed 4 address(es). 
| 17.172.239.128
| 192.168.0.2
| 239.255.255.250
|_192.168.0.8
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 10.20 seconds 

Optionally, these targets can also be added to the scanning queue on the fly:

#nmap -sL --script=targets-sniffer --script-args=newtargets -e <interface>
Starting Nmap 6.47SVN ( http://nmap.org ) at 2014-11-30 22:15 CST
Pre-scan script results:
| targets-sniffer: Sniffed 5 address(es). 
| 224.0.0.251
| fe80::7a31:c1ff:fec1:9c0a
| 192.168.0.8
| 192.168.0.2
|_239.255.255.250
Nmap scan report for 192.168.0.8
Host is up (0.0066s latency).
Not shown: 98 closed ports
PORT     STATE SERVICE
8008/tcp open  http
8009/tcp open  ajp13
MAC Address: 6C:AD:F8:7B:83:AB (Azurewave Technologies)

Nmap scan report for 192.168.0.2
Host is up (0.0033s latency).
Not shown: 99 closed ports
PORT      STATE SERVICE
49152/tcp open  unknown
MAC Address: 00:18:F5:0F:AD:01 (Shenzhen Streaming Video Technology Company Limited)

Nmap done: 4 IP addresses (2 hosts up) scanned in 16.01 seconds

Password auditing

Brute-force password-auditing scripts have grown to cover a lot of different services, thanks to the brute NSE library. This library allows NSE developers to easily launch dictionary attacks by implementing a simple class that uses other NSE libraries such as unpwd, which gives access to a username and password database. If any credentials are found during the execution, they will be added to a credentials database that can be read by other scripts.

Brute-forcing MySQL passwords

The mysql-brute script will help us perform brute-force password auditing against local or remote MySQL servers. In most configurations, MySQL will not impose a limit of login retries, so this is a good vector for exploiting weak passwords:

$nmap -p3306 --script mysql-brute <target>

If any credentials are found, they will be included in the script output:

3306/tcp open mysql
| mysql-brute: 
| root:<empty> => Valid credentials
|_ test:test => Valid credentials

Brute-forcing SMTP passwords

The smtp-brute script was written to help perform brute-force password-auditing attacks against SMTP servers, as the name states:

$nmap -p25 --script smtp-brute <target>

The output of this script is similar to that of other scripts that depend on the brute library:

PORT STATE SERVICE REASON
25/tcp open stmp syn-ack
| smtp-brute: 
| Accounts
| acc0:test - Valid credentials
| acc1:test - Valid credentials
| acc3:password - Valid credentials
| acc4:12345 - Valid credentials
| Statistics
|_ Performed 3190 guesses in 81 seconds, average tps: 39

Vulnerability scanning

NSE offers a great framework for penetration testers who need to create tools to detect and exploit vulnerabilities. Nmap offers a lot of options such as low-level packet creation and handling, libraries used to communicate with the most popular protocols, and an interface to report vulnerabilities. For those who don't need to write new tools but simply want to scan their network, there are very useful scripts to detect common misconfigurations and automate tedious tasks such as finding forgotten backup files and performing security checks.

Detecting insecure MySQL server configurations

The mysql-audit script inspects the configuration of your MySQL server against a list of security controls. This script requires that you set up some arguments:

$nmap -p3306 --script mysql-audit --script-args 'mysql-audit.username="<username>",mysql-audit.password="<password>",mysql-audit.filename=/usr/local/share/nmap/nselib/data/mysql-cis.audit' <target>

Each control in the database will be audited. The following are the results of a clean MySQL server installation on an Ubuntu server:

PORT STATE SERVICE 
3306/tcp open mysql 
| mysql-audit: 
| CIS MySQL Benchmarks v1.0.2 
| 3.1: Skip symbolic links => PASS 
| 3.2: Logs not on system partition => PASS 
| 3.2: Logs not on database partition => PASS 
| 4.1: Supported version of MySQL => REVIEW 
| Version: 5.1.41-3ubuntu12.10 
| 4.4: Remove test database => PASS 
| 4.5: Change admin account name => FAIL 
| 4.7: Verify Secure Password Hashes => PASS 
| 4.9: Wildcards in user hostname => PASS 
| 4.10: No blank passwords => PASS 
| 4.11: Anonymous account => PASS 
| 5.1: Access to mysql database => REVIEW 
| Verify the following users that have access to the MySQL database 
| user host 
| root localhost 
| root builder64 
| root 127.0.0.1 
| debian-sys-maint localhost 
| 5.2: Do not grant FILE privileges to non Admin users => PASS 
| 5.3: Do not grant PROCESS privileges to non Admin users => PASS 
| 5.4: Do not grant SUPER privileges to non Admin users => PASS 
| 5.5: Do not grant SHUTDOWN privileges to non Admin users => PASS 
| 5.6: Do not grant CREATE USER privileges to non Admin users => PASS 
| 5.7: Do not grant RELOAD privileges to non Admin users => PASS 
| 5.8: Do not grant GRANT privileges to non Admin users => PASS 
| 6.2: Disable Load data local => FAIL 
| 6.3: Disable old password hashing => PASS 
| 6.4: Safe show database => FAIL 
| 6.5: Secure auth => FAIL 
| 6.6: Grant tables => FAIL 
| 6.7: Skip merge => FAIL 
| 6.8: Skip networking => FAIL 
| 6.9: Safe user create => FAIL 
| 6.10: Skip symbolic links => FAIL 
| 
|_ The audit was performed using the db-account: root 

Detecting web servers vulnerable to slow denial-of-service attacks

Slow denial-of-service attacks open as many connections as possible and send the minimum amount of data, taking the longest possible time to attempt to consume all available network resources. The http-slowloris and http-slowloris-check scripts allow the detection of web servers vulnerable to these attacks. Robert Hansen, better known as "RSnake," has published a tool and documented this vulnerability very well at http://ha.ckers.org/slowloris/. Also, a security researcher named Hugo Gonzalez discovered that these attacks can be ported to IPv6 as well.

Running the http-slowloris script with a high number of concurrent connections will launch a slow denial-of-service attack:

#nmap -p80 --script http-slowloris --max-parallelism 300 <target>

If the host is vulnerable, the output will return something similar to this:

PORT STATE SERVICE REASON 
80/tcp open http syn-ack
| http-slowloris:
| Vulnerable:
| the DoS attack took +5m35s
| with 400 concurrent connections
|_ and 1900 sent queries

Detecting SSL servers vulnerable to CVE-2014-3566

The vulnerability known as CVE-2014-3566, also known as Poodle, allows decryption of secure communications using SSL version 3. Although there are newer security protocols, downgrade attacks can be performed on modern web browsers to force connections to fall back to SSLv3. Therefore, SSLv3 is considered obsolete and insecure now.

To detect services that allow SSLv3 CBC ciphers, we could use the ssl-poodle NSE script:

nmap -sV --version-all --script ssl-poodle -p- <target>

Vulnerable services will return the following output:

PORT    STATE SERVICE REASON
443/tcp open  https   syn-ack
| ssl-poodle:
|   VULNERABLE:
|   SSL POODLE information leak
|     State: VULNERABLE
|     IDs:  CVE:CVE-2014-3566  OSVDB:113251
|           The SSL protocol 3.0, as used in OpenSSL through 1.0.1i and
|           other products, uses nondeterministic CBC padding, which makes it easier
|           for man-in-the-middle attackers to obtain cleartext data via a
|           padding-oracle attack, aka the "POODLE" issue.
|     Disclosure date: 2014-10-14
|     Check results:
|       TLS_RSA_WITH_3DES_EDE_CBC_SHA
|     References:
|       https://www.imperialviolet.org/2014/10/14/poodle.html
|       http://osvdb.org/113251
|       http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-3566
|_      https://www.openssl.org/~bodo/ssl-poodle.pdf
 

Setting up a development environment


To start developing NSE scripts, you don't need anything but a fresh copy of Nmap and your favorite text editor (vi, nano, gedit, and so on). However, you need to configure your text editor to use two space indents instead of tabs if you are planning on sending your contributions to the development mailing list.

There is a file named HACKING in your Nmap installation directory that you should read. It contains useful tips for people interested in NSE development. If you are working with vi, you might want to add the following to your .vimrc file. It contains a couple of additions to the rules listed in the HACKING file:

syntax enable
au BufRead,BufNewFile *.nse set filetype=lua
set nocindent
set expandtab
set softtabstop=2
set shiftwidth=2
set copyindent

Note

You can also download the file from my GitHub repository at https://github.com/cldrn/nmap-nse-scripts/blob/master/.vimrc.

Halcyon IDE

For those who love working with graphical environments, there is an unofficial IDE, named Halcyon IDE, created exclusively to develop NSE scripts. It is written in Java and allows developers to test and debug scripts within itself, providing features such as code completion and syntax highlighting. The following screenshot shows the Halcyon IDE:

The development of this IDE is still in its early stages so I recommend submitting any bugs you encounter. The official GitHub repository can be found at https://github.com/s4n7h0/Halcyon.

 

Adding new scripts


NSE scripts are listed in a file named script.db. Having your NSE scripts included in this database allows you to call them directly by name (without the .nse extension). To add new scripts to your script.db database, you simply need to copy your .nse files to the scripts directory, which is usually <NMAP install>/scripts, and run the following command:

#nmap --script-updatedb
 

Summary


In this chapter, we introduced NSE and its amazing capabilities. By now, you should have installed the latest version of Nmap and have your development environment ready to go. The Nmap options covered in this chapter will be all you need to comfortably run and debug NSE scripts. Pay close attention to the different script rules (prerule, postrule, portrule, and hostrule) that will be shown throughout the book.

Now we are ready to start writing NSE scripts and get familiar with all the available libraries. In the following chapters, you will discover the true power of NSE. The next chapter covers the fundamentals of Lua programming, so prepare yourself to learn this amazing scripting language.

About the Author

  • Paulino Calderon

    Paulino Calderon (@calderpwn on Twitter) is the cofounder of Websec, a company offering information security consulting services based in Mexico and Canada. When he is not traveling to a security conference or conducting on-site consulting for Fortune 500 companies, he spends peaceful days in Cozumel, a beautiful small island in the Caribbean, learning new technologies, conducting big data experiments, developing new tools, and finding bugs in software.

    Paulino is active in the open source community, and his contributions are used by millions of people in the information security industry. In 2011, Paulino joined the Nmap team during the Google Summer of Code to work on the project as an NSE developer. He focused on improving the web scanning capabilities of Nmap, and he has kept contributing to the project since then. In addition, he has been a mentor for students who focused on vulnerability detection during the Google Summer of Code 2015 and 2017.

    He has published Nmap 6: Network Exploration and Security Auditing Cookbook and Mastering the Nmap Scripting Engine, which cover practical tasks with Nmap and NSE development in depth. He loves attending information security conferences, and he has given talks and participated in workshops in dozens of events in Canada, the United States, Mexico, Colombia, Peru, Bolivia, and Curacao.

    Browse publications by this author