ModSecurity 2.5: Prevent web application hacking with this easy to use guide

ModSecurity version 2.0 was first released in October 2006 and is a big improvement over the previous major version line. It is a substantial rewrite that changes the rule language and adds a lot of improvements such as:

At the time this book was published, the latest major ModSecurity version was 2.5, and this version line adds even more enhancements such as the ability to reference a geographical database, which allows you to create rules that take action based on the geographical location of the user. Another interesting new feature is credit card number detection, which can be used to detect and prevent credit card numbers from being exposed through your web server. Of course, all the other security features that make ModSecurity such a great web application firewall have been refined and are available in the latest version, and we will learn all about them in the coming chapters.

Since version 2 of ModSecurity is such a different beast to previous versions, this book focuses only on this latest major version branch. This means that you must run Apache 2.0 or later, as ModSecurity 2 requires this Apache branch to function.

As Apache 1.x is a legacy branch that is now only infrequently updated (and when updated, mostly to patch known security vulnerabilities), now might be a good time to upgrade to the 2.x branch of Apache if you're still running an older version.

ModSecurity was originally developed by web application security specialist Ivan Ristic in 2002. He has also written the excellent book Apache Security (O'Reilly Media, 2005) which I highly recommend if you want a general book on hardening Apache. ModSecurity was acquired by Breach Security, a California-based web application security company, in 2006. The company chose to continue releasing ModSecurity as a free open source product (under the GPLv2 license), hiring Ivan and additional staff to work on the product, which is good news for all users of ModSecurity.

The ModSecurity source code is available at http://www.modsecurity.org/download/. The source is provided as a .tar.gz archive—to download it all you have to do is copy the link to the latest release and you will then be able to use wget to download the source archive to your server.

In the following text the name of the file used for the source archive is assumed to be modsecurity‑apache.tar.gz. Make sure you substitute the actual file name or web location (which usually includes the version number of the latest release) for this name when downloading or working with files.

$ wget http://www.modsecurity.org/download/modsecurity-apache.tar.gz
Resolving www.modsecurity.org... 216.75.21.122
Connecting to www.modsecurity.org|216.75.21.122|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://downloads.sourceforge.net/mod-security/modsecurity-apache.tar.gz?use_mirror= [following]
[...]
HTTP request sent, awaiting response... 200 OK
Length: 1252295 (1.2M) [application/x-gzip]
Saving to: `modsecurity-apache.tar.gz'
[...]
'modsecurity-apache.tar.gz' saved [1252295/1252295] ]

Checking the integrity of the downloaded source archive

Checking the integrity of the downloaded archive file is always a good habit. This ensures that the file has not been tampered with in any way. There are two ways to do this—a less secure and a more secure way. The less secure way is to use the md5sum tool to calculate the MD5 sum of the downloaded file and then compare this MD5 sum to the one published on the ModSecurity website.

MD5 is an algorithm of a type called "cryptographic one-way hash". It takes an input of an arbitrary size (the source archive, in this case), and produces an output of a fixed length. A hash function is designed so that if even one bit changes in the input data, a completely different hash sum is calculated. The hash function should also be collision resistant. This means that it should be very hard to create two files that have the same hash value.

Using the MD5 sum to verify the integrity of the archive is less than optimal for two reasons: :

1. If anyone had the ability to alter the source code archive then they would also have the ability to alter the file that contains the calculated MD5 sum and could easily make the bad source distribution appear to have a valid checksum.

2. The other, and less subtle reason to not use the checksum approach, is that it was recently discovered that the MD5 checksum function is not collision resistant. In 2008, a group of researchers used 200 Sony PlayStation 3 consoles (yes, really!) to create a falsified web server certificate using attacks on the MD5 function. All in all, this means that the MD5 checksum function is no longer considered secure.

The better way to verify the integrity of the downloaded source archive is to use public key cryptography. In public key cryptography, encryption and decryption are performed using different keys. Encryption is performed using a private key, which only the person encrypting a file or document has access to. Decryption is done using a public key, which anyone can access and which can be published online.

When a file is signed using public key cryptography, a checksum for the file is first calculated, just like with the MD5 algorithm described above. The calculated hash is then encrypted using the signer's private key. You can then verify the integrity of the signed file by decrypting the hash (using the signer's public key) and comparing it to the calculated hash value. All of this is done automatically using a program such as PGP or Gnu Privacy Guard (GPG).

The developers of ModSecurity have signed the source code archive using their private key, which allows us to verify its integrity in the manner just described. The first thing we need to do in order to verify the archive is download the file that contains the signature:

$ wget http://www.modsecurity.org/download/modsecurity-apache.tar.gz.asc

We can then use the open source program GPG to verify the signature. GPG comes pre-installed on most Linux systems; however should the program not be installed on your system you can get it at http://www.gnupg.org.

When we try to verify the signature of the source archive using GPG we will encounter a problem, as we don't have the public key of the person who signed the file:

Fixing this is however easy. All we need to do is download the public key file used to sign the file, as specified by the key ID in the output above. The key is available on the server pgp.mit.edu, which is a repository of public key files.

Tip

If you have a firewall controlling outbound traffic, you need to enable connections to remote port 11371 for GPG to be able to download the key.

The following command is used to download the key from the server:

Now that we have downloaded the public key, all the required elements to check the signature are in place. Running the verification command again produces this output:

The verification of the source archive using the public key we just downloaded has succeeded, as evidenced by the line starting with Good signature from. However, what about the ominous-looking message Warning: This key is not certified with a trusted signature?

Public key cryptography tools such as GPG work using a concept called web of trust. In the same way that you might trust that your best friend's parents are the people he introduces to you as his parents, a public key can be trusted if other people you trust have verified that the key belongs to the actual person it is issued to. This verification of another key is called signing the key, and this can be done by many people (to continue our analogy, this would be like other people verifying that your best friend's parents are the people he introduced you to).

If you don't already have public keys installed on your system that build a chain of trust and verify that the key you just used really does belong to Brian Rectanus, there is a (very small) chance that someone could have forged his public key. Fortunately, for those who are very paranoid, or are working on a project that has high security demands, it is possible to verify that a public key belongs to a person. This is done by taking the key's fingerprint, and asking someone who knows Brian (or even Brian himself) to verify that his key has the fingerprint shown on your copy. You can show the fingerprints of all the keys you have imported into GPG by executing gpg --fingerprint.

If you have downloaded the gzip file with the source code and saved it as modsecurity-apache.tar.gz you can use the following command to unpack it:

$ tar xfvz modsecurity-apache.tar.gz

This will unpack the source code into a subfolder with the name modsecurity-apache. It will also create a directory structure in this folder where the different subfolders will hold the source code, documentation, and sample rules, among other things. A typical layout of the directories is as follows:

ModSecurity requires the following additional components before you can compile it:

apxs is the APache eXtenSion tool and is used to compile extension modules for Apache. Since ModSecurity is an Apache module this tool is required to be able to compile ModSecurity. You can see if you have apxs installed on your system by running the following:

$ whereis -b apxs

If apxs is available the above command will return its location, like so:

$ whereis -b apxs
apxs: /usr/sbin/apxs

If you don't have apxs installed then it is available as part of a package called httpd-devel (or apache2-dev on Debian, Ubuntu, and related distributions). Use your favorite package manager to install this and you should then have apxs available on your system.

libxml2 is an XML parsing library. If you don't have this installed then you can get it by installing the package libxml2-devel (or libxml2-dev if you're using a Debian-based distribution).

Finally, mod_unique_id is an Apache module that generates a unique identifier for each HTTP request. (See http://httpd.apache.org/docs/2.0/mod/mod_unique_id.html if you are interested in the technical details on how this works.) Apache usually comes with this module pre-compiled, but you'll need to insert the following line in the module list of httpd.conf (you can find this list by looking for a bunch of lines all starting with the LoadModule directive) and restart the server for the module to be activated:

LoadModule unique_id_module modules/mod_unique_id.so

Note that this procedure for enabling the module is for Red Hat/Fedora-based distributions. On Debian/Ubuntu, for example, you would use the command a2enmod unique_id to enable the module.

To verify that mod_unique_id is indeed loaded into Apache you can run the following command and check for the presence of the line unique_id_module (shared) in the output listing:

$ httpd -t -D DUMP_MODULES
…
unique_id_module (shared)

As with other Linux software that comes as source, you need to compile ModSecurity to be able to use it. Compilation will result in a file called mod_security2.so, which is a binary shared module used by the Apache server in a plugin-like fashion. This module file contains all the functionality of ModSecurity.

To get started compiling the source, change to the root user as you will require root privileges to install ModSecurity. Then change to the apache2 subfolder of the directory where you unpacked ModSecurity (for example, /home/download/modsecurity-apache/apache2/). This directory contains the source files and all the files needed to build the binary module.

To be able to compile the binary, you need a Makefile, which is a file that contains details of your particular server setup such as which compiler is available and what options it supports. To generate the Makefile, run the following command:

[apache2]$ ./configure
...
config.status: creating Makefile
config.status: creating build/apxs-wrapper
config.status: creating mod_security2_config.h

If the configure script stops with an error indicating that the PCRE library cannot be found, this is usually because you have compiled Apache from source and it has used the PCRE library that is bundled with the Apache distribution. Running configure --with-pcre=/path/to/apache-src/srclib/pcre should solve the problem (if it doesn't, edit Makefile and change the PCRE_CFLAGS and PCRE_LIBS variables to point to the pcre directory).

After this command has completed, check for the presence of a file called Makefile in the current directory. After making sure it exists you can go ahead and compile the binary:

[apache2]$ make

You should see a fairly long list of messages written to the terminal as the compilation takes place, and if everything goes well there should be no error messages (though you may get a few compiler warnings, which you can ignore).

The compilation process outlined in the previous section results in a file called mod_security2.so being created. This is an Apache dynamic shared object which is a plugin to Apache that adds functionality to the web server without requiring it to be recompiled. This file contains all the ModSecurity functionality, and integrating it like any other Apache module is, except for some basic configuration, all it takes to enable ModSecurity on your server.

The mod_security2.so file is output to the modsecurity-apache/apache2/.libs directory by the compiler. To let Apache know about ModSecurity, start by copying the mod_security2.so file to your Apache modules directory. Typically the modules directory will be something like /etc/httpd/modules, but the location will vary depending on your setup.

The next step is to edit the Apache configuration file and add a line to let the web server know about the new module. Start your favorite editor and open up httpd.conf (again, the location will vary depending on your setup, but assuming the same Apache base directory as in the previous section, the file will be in /etc/httpd/conf/httpd.conf). It's a good idea to create a backup copy of httpd.conf before you start editing the file, so that you can revert to the backup if anything goes wrong.

In httpd.conf there will be a fairly long list of configuration directives that start with the word LoadModule. Find this section of LoadModule directives and add the following line to the top of the list:

LoadModule security2_module modules/mod_security2.so

The security2_module string is known as the module identifier, and is declared in the source code of each module. It is used by Apache to later identify the module in such directives as IfModule, which turn on or off processing of configuration directives based on whether or not the module is loaded.

After adding this line, exit the editor and run apachectl configtest. This will test the new configuration file and report back any errors so you can fix them before attempting to restart the server. If all went well, run apachectl restart to restart the web server. This will load ModSecurity which means the fun part of writing rules can soon begin!

It is best to put all the configuration and security rules for ModSecurity in a separate file in the conf.d sub-directory of the Apache root directory. This prevents you from cluttering your main Apache configuration file with ModSecurity directives.

Simply start your favorite editor, create a file called modsec.conf in the conf.d directory, and enter the following to get started:

<IfModule security2_module>
# Turn on rule engine and set default action SecRuleEngine On SecDefaultAction "phase:2,deny,log,status:403"
</IfModule>

Make sure the IfModule directive uses the module identifier you provided in the LoadModule line in httpd.conf (security2_module in this case), otherwise Apache will ignore everything between the start and end of IfModule.

SecRuleEngine On turns on the rule engine so that it will start processing rules. For debugging purposes you can also set this to Off (which will turn off rule processing) or DetectionOnly, which will process rules but not take any action, even if a rule matches (which is helpful if you want to test that rules are working, but not block any requests should there be a problem with the rules).

The SecDefaultAction line above specifies what happens when a rule match occurs. In this case we want ModSecurity to deny the request with a status code 403 ("Forbidden"), and to write a log entry (which will show up in the Apache error log and the ModSecurity audit log). The default action is to allow requests even if a rule matches, so it is important to add this line to make sure any matching rule results in the request being denied.

You may be wondering what the phase:2 statement in the above directive does. ModSecurity divides the processing of a request into five phases—request headers, request body, response headers, response body and logging:

As can be seen by the table, the most useful phase when we want to inspect incoming HTTP requests is the request body phase, in which all of the request headers, as well as the body, are available. By specifying phase:2 for the default action, subsequent rules will all be processed in phase 2 unless another phase is specified in a rule.

To override the default phase for a rule, you use the phase directive, as can be seen in this example, which stops processing and denies the request if the request header's user-agent field contains the string WebVulnScan, which is a script to find weaknesses in web servers:

SecRule REQUEST_HEADERS:User-Agent "WebVulnScan" "phase:1"

This will cause the rule to be processed in phase 1—after the request headers have been received.

<IfModule security2_module> # Turn on rule engine and set default action
SecRuleEngine On SecDefaultAction "phase:2,deny,log,status:403"
# Configure request body access and limits
SecRequestBodyAccess On
# Debug log settings
SecDebugLog logs/modsec_debug.log
SecDebugLogLevel 0 </IfModule>

After completing the installation we need a way to test that the ModSecurity module has been loaded and is working as it should. The procedure described here can be used to test that ModSecurity is functioning correctly whenever you feel the need to verify this (such as after making changes to your Apache configuration file).

$ echo "The owl flies at midnight" > secret.html

# Block all requests that have the string "secret" in the URI SecRule REQUEST_URI "secret"

# Send full server signature so ModSecurity can alter it ServerTokens Full

# Alter the web server signature sent by Apache SecServerSignature "Apache 1.3.24"

$ echo -e "HEAD / HTTP/1.0\n\n" | nc localhost 80
HTTP/1.1 200 OK Date: Wed, 28 Jan 2009 15:01:56 GMT
Server: Apache 1.3.24
Last-Modified: Mon, 26 Jan 2009 12:01:12 GMT
ETag: "6391bf-20-461617eaf9200"
Accept-Ranges: bytes
Content-Length: 32
Connection: close
Content-Type: text/html; charset=UTF-8

These simple examples have hopefully given you a taste of what can be accomplished with ModSecurity. There are many powerful functions left, and you may be amazed at some of the things that can be done using this versatile module.

In this chapter we first looked at downloading the ModSecurity source code and verifying its integrity. We then compiled the source code to produce the binary module file mod_security2.so. After this, we integrated the module file with Apache, and tested the new setup to make sure that ModSecurity was properly installed and working. Finally, we used ModSecurity to alter Apache's server signature by employing the SecServerSignature directive.

We are now done with the installation and basic setup of ModSecurity. In the next chapter we move on to learning all about writing rules.