Securing Network Services with FreeBSD Jails

In this article by Christer Edwards, we will explore FreeBSD Jails. FreeBSD Jails are a kernel-level security mechanism which allows you to safely segregate processes within a sandbox environment. Jails are commonly used to secure production network services like DNS or Email by restricting what a process can access. In the case of a malicious attack on one service, all other Jailed processes would remain secure. FreeBSD Jails securely limits, in an administratively simple way, the amount of damage an attacker can do to a server.

Introduction

Is it possible to easily run a half-dozen internet services on a single piece of hardware and make sure that if one is compromised the others will remain unharmed? Can this be done without a mountain of administrative overhead and customization? Can I configure my services the way I have grown accustomed? Absolutely! This article will outline how to achieve this, through the use of FreeBSD Jails.

Over the course of this article I will outline how to install a list of production services on a single piece of hardware, securing each one from the next, all with only one additional administrative tool: ezjail

Before we get to the ezjail tool we need to define FreeBSD Jails. What are they? What do they do? Why do I care?

FreeBSD Jails are a kernel-level security tool used widely in the FreeBSD community to segregate processes. An easy way to think of a Jail is that it is very much like a chroot environment, but much more hardened. While a standard chroot environment can often be escaped, FreeBSD has added code to their kernel which hardens the chroot environment into a "Jail"—Inescapable. Within this Jailed environment processes are unable to identify, access or otherwise communicate with processes on the outside of the Jail. Networking is limited within the Jail as well. A Jail cannot affect any underlying network configuration other than that which it has been assigned. A Jail can also be thought of in many ways like a virtualized machine in that the virtual "guest" cannot interact with the physical "host". Jails allow us the opportunity to run processes in a secure manner separate from our host environment.

If that sounds appealing to you may be wondering how to activate and use this Jail system. That, my friend, is the focus of this article. Get settled because by the time we're done here you will have all the tools you need to segregate processes for security, sandboxing or even create custom environments for other users.

By default the Jail system is part of the FreeBSD kernel. The kernel customizations to make the system possible have such a minimal footprint that it was decided it should be a default, always-on feature of FreeBSD. Your FreeBSD installation already has the ability to do everything described above, you just need to know how to use it.

Ezjail

The tool that I use to create, manage and interact with my FreeBSD jails is called "ezjail". It simplifies much of the underlying configuration of a Jail system to the extent that you can create a Jail and be working within it in just three steps!

To install the ezjail port you need to make sure you have your ports tree updated and then run:

cd /usr/ports/sysutils/ezjail && make install clean

Before we can create any Jails we'll need to create the base Jail environment. This is the template environment from which all other Jails will be created. This is simplified by the ezjail-admin tool:

ezjail-admin install

This will download the components of a base Jail system. Also, the -m, -s and -p options install the man pages, source packages and ports tree respectively. If you want access to these within your Jail environments then be sure to append them to the command above.

Before any Jails will be able to start you'll also need to activate the ezjail system within the /etc/rc.conf. This is done using the command:

echo 'ezjail_enable="YES"' >> /etc/rc.conf"

Now that you've got the proper tools installed you'll need to keep in mind the following requirements for creating and maintaining a Jail environment. These items will need to be defined for each Jail environment that you want to create, and will need to persist for the life of the Jail.

• jailname
• IP Address(es)
• custom jailroot (storage directory) - optional

For our purposes here we'll create three Jail environments. We'll call them "larry", "curly" and "moe". Each of these will be assigned an IP address on our internal LAN using "192.168.0.100", "192.168.0.101" and "192.168.0.102" respectively. We will also use the default jailroot path, which is /usr/jails/jailname/.

Based on the above decisions we have three of the items defined, but only two configured. We will need to create interface aliases for our network device which will bind to the addresses we've decided on. There are two ways this can be done. The first method, the temporary method, will work for quickly testing Jails and creating environments that you don't need to keep. The second method, the persistent method, will define the interface aliases in your system configuration and define them persistently across reboots. The persistent method is what you will need if you plan on continuing to use your Jails long term.

Note: FreeBSD defines network interfaces by their device name or module name. Replace hme0 with your interface name as required.

Temporary Network Alias

To create temporary network aliases for the three Jails you would run these commands (replacing each IP as needed):

ifconfig hme0 alias 192.168.0.100 netmask 255.255.255.255
ifconfig hme0 alias 192.168.0.101 netmask 255.255.255.255
ifconfig hme0 alias 192.168.0.102 netmask 255.255.255.255

Persistent Network Alias

To create persistent network aliases (aliases that will persist across reboots) you would add the following to your /etc/rc.conf file (replacing your IP as needed):

ifconfig_hme0_alias0="inet 192.168.0.100/32"
ifconfig_hme0_alias1="inet 192.168.0.101/32"
ifconfig_hme0_alias2="inet 192.168.0.102/32"

Creating a Jail environment

Once you have activated your network aliases and the third and final configuration requirement is met we're ready to create these Jails. You can create a Jail environment using the command below. Repeat for each Jail, replacing jailname and jailip as needed:

ezjail-admin create jailname jailip

In this situation we would have run the following commands:

ezjail-admin create larry 192.168.0.100
ezjail-admin create moe 192.168.0.101
ezjail-admin create curly 192.168.0.102

You will see a bunch of output on your screen. This is normal. The output shows that files are being put into place and underlying configuration is happening. Remember, without the ezjail-admin tool you'd need to do that configuration by hand. No thanks!

Your Jail environments are now ready to use! Wasn't that easy! To move from you host system to your Jail environment simply use the command:

ezjail-admin console jailname

This command will give you a console connection into the Jail environment. It will act just as if you had sat down and logged into the machine (although no login credentials are required). You should now be within one of your Jails, logged in as root, with a base FreeBSD system. No ports are installed. Nothing is configured. None of the host customizations are adopted. You have a pristine, minimal FreeBSD installation to begin building your services.

Configure and activate SSH

Let's configure a service and make this Jail more accessible. First, SSH: SSH is part of the base FreeBSD installation so all we need to do is configure and activate the service. It should work out of the box, but it can't hurt to take a look at the configuration for the SSH daemon, located in /etc/ssh/sshd_config.

You may want to update the following lines:

ListenAddress 0.0.0.0

For this example we'll change the port to 2200 and the ListenAddress to 192.168.0.100.

Update the Jail environment to launch the SSH daemon at startup by adding the following line to your /etc/rc.conf:

sshd_enable="YES"

Finally, start the service manually by running:

/etc/rc.d/sshd start

If you now logout of your Jail (type "exit") and take a look at your host system using netstat you should find that it is listening on 192.168.0.100:2200—the host address and port (assuming ssh is configured on the host system). You can find this information using:

netstat -nat | less

Building, securing, and maintaining networks with the FreeBSD operating system Set up and manage networking on FreeBSD Virtualization with FreeBSD Jails, IPFW and PF Configure interfaces, protocols, and routing  http://www.packtpub.com/network-administration-with-freebsd/book

Configure Jail to run Web Server

Let's now configure a second Jail to run a web server.

ezjail-admin console moe

Again you should find yourself at a prompt within a base system. The changes you made within the first Jail are not present in the second. You might even run netstat again and see if you can find the listening service we just setup. What do you bet? You don't see it at all!

Now we'll install our web server. In this case we'll install Lighttpd, as it is my current favorite.

cd /usr/ports/www/lighttpd && make install clean

When that port is finished installing we can fire it up and compare the netstat output from each environment. You should find that:

• The Host environment lists all listening addresses and ports, even those housed within a Jail
• The first Jail only lists SSH on port 2200
• The second Jail only lists Lighttpd on port 80

Are you impressed so far? The whole process is probably simpler than you expected. One of my favorite aspects of the whole Jail system is that, outside of learning a few simple ezjail-admin commands, everything else is as it would be on a normal system. I don't have to change any configuration. I don't have to customize my services to run in a special environment. I can treat my Jail environments the same way I treat any other environment, except I know that if my Jail is compromised it will be the limit of the damage. If someone malicious were to break into your SSH Jail, your web server would remain unharmed and continue to serve content to the public. Even if the attacker were able to gain root privileges, the extent of the damage would be limited to the services only and the configuration is securely contained within that Jail.

Configure Jail as mail server with Postfix

Thirdly we'll configure our last jail as a mail server with Postfix. Exit your second Jail and console into the third. Install Postfix using:

cd /usr/ports/mail/postfix && make install clean

You'll want to pay attention to the final output of the compilation. It mentions a few files to customize in replacing the default Sendmail with Postfix. Again, these customizations are not unique to a Jail environment but are standard across any FreeBSD installation.

You'll want to change one line in the Postfix configuration, which will tell the service to listen on its public interface rather than just localhost. Open the config file in /usr/local/etc/postfix/main.cf and search for the setting for "inet_interfaces". Change it to "inet_interfaces= all".

Make sure Postfix is configured to start by adding the following line to your /etc/rc.conf:

postfix_enable="YES"

And lastly we'll start the service.

/usr/local/etc/rc.d/postfix start

Network level separation

If you're still curious about the network level separation between Jails you can run the netstat command again (netstat -nat) and verify that you only see the Jailed services within the appropriate Jails. I think you get the idea from the last two attempts though. These services are completely cut off from the rest. The only place that you're able to see them is within the appropriate Jail itself or within the host system.

I'll leave further configuration of these services up to a future article, but take a step back and take a look at what we've created here. We now have four logical systems. One is managing the other three (in that regard it is similar to a virtual machine hypervisor) and the other three are running public services. We've created an SSH machine which listens on a non-standard port. We've created a web server and we've setup the basics of a mail server. All within one piece of hardware and all with a minimal performance hit. Take a look at your load average on the host system.

What do you see? I bet it is minimal.

Now that we've established that network services are contained within our Jails why don't we take a look at the local processes underneath. Are the processes really hidden from one Jail to the next? A Jail is similar to a chroot environment, right? Can it really limit access to underlying processes as well? Let's take a look.

Disconnect from the third Jail system and make sure you have a shell on the host system. Let's start 'top' and see what we find listed there. You should see all of the processes of the host system, plus the services running within the Jails. Now let's do the same within one of the Jails.

Console into your first Jail environment again using the command:

ezjail-admin console larry

Run 'top' again. What do you see?  It's quite a bit less. You should see ssh, syslog, and maybe cron. Not much else. You definitely don't see Postfix or Lighttpd. The Jail environment you are in is limited in the processes that it can list. You are only able to start, list or stop processes that are running within that Jailed environment.

Let's try something a little more specific. Exit out of that Jail environment and run top again. Try to find Postfix or Lighttpd in the list and make note of the PID (process ID). Now console back into your first Jail and try to kill that process.

kill <PID>

What happens? Are you able to kill a Jailed process from within a different Jail? No. You should get an error along the lines of:

- No such process

Even though we've verified that the process is running, and even though we have all the powers of the root user we are unable to kill that process. It is outside of the confines of our Jailed environment and based on that we have no access or control over it.

Just imagine that restriction in the case of a malicious attack. An intruder gains access to one of your Jailed environments. He has no way of knowing that he is confined to a Jail and starts to wreak havoc. He could kill off every process that he can find within the Jail, but even if he attempted to kill process IDs outside of the Jail he would be powerless. Imagine that. The mighty root user has a limit to what he can do!

Now for a few final things before we wrap up.

Options of ezjail-admin command

We've used a few of the ezjail-admin commands. What are some of the other options? Let's take a look.

At the time of this writing ezjail-admin is at version 3.0. The options available to this version are listed below.

ezjail-admin v3.0
Usage: ezjail-admin [archive|config|console|create|delete|install|list|restore|update] {params}

I will quickly define what these options do. More information can be found in the main page for ezjail-admin.

• ezjail-admin archive: Creates a backup of one, multiple or all ezjails.
• ezjail-admin config: Manages specific jails. (autostart, no autostart, rename an ezjail).
• ezjail-admin console: Attaches your console to a jail by executing a jexec with its jid.
• ezjail-admin create: Installs a new jail defined by jailname and jailip.
• ezjail-admin delete: Removes a jail from the configuration and disables startup (Jail files still remain).
• ezjail-admin install: Fetches everything needed to setup a Jail environment via FTP and installs it (required before ezjail-admin create).
• ezjail-admin list: List all Jails, jailips and jaildirectories.
• ezjail-admin restore: Creates new Jails from archived versions (see ezjail-admin archive).
• ezjail-admin update: Updates basejail environment. Basejail is the source template for all Jail environments.

Summary

In conclusion, the FreeBSD Jail system is a very powerful tool for securing network services and system processes. From a security perspective, and keeping usability and administrative overhead in mind, it is a simple yet powerful tool in the System Administrators arsenal. I am currently running seven Jails within my home LAN, each running a different network-based service. I feel confident knowing that a security breach to any one of the Jails still leaves the rest of my system safe. Also, by using the ezjail tool, I can very quickly create and destroy sandbox environments for testing without affecting my production services. I invite you to learn more about FreeBSD Jails by studying the FreeBSD handbook on the topic found here. [http://www.freebsd.org/doc/en/books/handbook/jails.html]

If you have read this article you may be interested to view :

• Install GNOME-Shell on Ubuntu 9.10 "Karmic Koala"
• Ubuntu 9.10: How To Upgrade
• Securely Encrypt Removable Media with Ubuntu
• Ubuntu User Interface Tweaks
• Create a Local Ubuntu Repository using Apt-Mirror and Apt-Cacher

Building, securing, and maintaining networks with the FreeBSD operating system Set up and manage networking on FreeBSD Virtualization with FreeBSD Jails, IPFW and PF Configure interfaces, protocols, and routing  http://www.packtpub.com/network-administration-with-freebsd/book

About the Author

Christer Edwards currently works for a US Department of DefenseContractor maintaining Linux and FreeBSD Systems. In a previous lifehe was a Technical Instructor developing and teaching courseware forRed Hat Enterprise. While it is the Enterprise  that pays the bills,his real passion is in the Linux community trenches. He has been along-time contributor to Ubuntu where he has maintained a technicalblog on the Ubuntu Planet for over three years. He was also key inorganizing the Ubuntu LoCo project throughout the US. An ideal day forhim is one spent quietly writing documentation and articles.