Linux Email: Set up and Run a Small Office Email Server

By Alistair McDonald , Carl Taylor , David Rusenko and 3 more
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Linux and E-Mail Basics

About this book

Many businesses want to run their email servers on Linux, but getting started can be complicated. The attractiveness of a free-to-use and robust email service running on Linux can be undermined by the apparent technical challenges involved.  Some of the complexity arises from the fact that an email server consists of several components that must be installed and configured separately, then integrated together. Unlike other approaches that deal with one component at a time, this book gives you a basic knowledge across all the server components, leaving you with a complete working email server for your small business network.

Based entirely on free, Open Source software, you will see how to protect your server from spam and viruses, offer web access for remote access, and secure your installation with regular backups.

Publication date:
July 2005
Publisher
Packt
Pages
316
ISBN
9781904811374

 

Chapter 1. Linux and E-Mail Basics

If you are one of those thousands of system administrators who manage the networks and computers of small to medium-sized companies thinking of hosting their own e-mail service, this book is for you.

We will start with the most basic components of an e-mail system. Together they will allow your users to receive mail from each other and from other people on the Internet, and send messages to whomever they want. This might be all you need, but many companies also want to provide their users with an accessible webmail service that people can use from home or when they are on the road. Another feature that many people unfortunately cannot be without today is proper protection against viruses spread via e-mail as well as filtering of spam messages.

This book will not cover all aspects of the software in question, but it will give you a solid ground to stand on and from there you will be able to delve into the more detailed descriptions of more advanced setups found elsewhere.

As the technical platform for our endeavor, we have chosen the GNU/Linux operating system and a proven selection of free software tools that will help us achieve the goal of a reliable e-mail server for the smaller company. The tools we have chosen are widely known and accepted, written by software professionals, and are supported by a large community of users.

In this very first chapter of the book, we start with what you need to know before you even start working on your server. Questions such as what kind of server hardware and network connection you need will be answered. We will also give you a brief introduction to internet e-mail and its working.

Why Manage your own E-Mail Server?

Why have your own e-mail server and manage it by yourself? Is that not why there are Internet Providers? Since you are, after all, reading this book you may already have your reasons, but let us anyway discuss these questions and some possible answers to them.The most important reason for hosting and managing your own e-mail server is control. For many organizations, e-mail is an important part of the IS/IT infrastructure. Keeping the control over your e-mail has many advantages:

  • If your company has offices in multiple places, you have full freedom when choosing how to connect them. Virtual private network between the offices, TLS-secured connections between the offices, a single server for all offices, one server per office, etc.

  • By keeping your own messaging in-house, you can send messages to each other without having them travel across the line to and from your provider. This gives a more reliable service if your Internet connection fails, and it avoids unnecessary latencies.

  • You are not dependent on the competence of the provider's staff. If you manage your own server and need to solve a difficult problem or implement a custom solution for something, just hire a consultant to help you.

  • If the provider goes bankrupt, all your data resides safely in your server room and on your backup media.

  • You are not subject to the limitations that your provider may set regarding, say, use of disk space or the maximum size of messages.

  • You can implement any policies for anti-spam or anti-virus that you choose.

More control requires more responsibility and more knowledge, and that is where this book comes in.

These hopefully compelling arguments aside, there are also downsides to hosting your own e-mail server. This is a task that requires a certain level of knowledge and commitment and so should not be undertaken be everyone. With your own server, you are not only responsible for the service you provide to your users, but you also have a responsibility towards the whole Internet community. An ill-configured e-mail server can help worms and spam to spread, which not only is a disservice to the community but can also get your server blacklisted. Even though a properly set up server can run for years without requiring much maintenance, you must keep yourself reasonably updated and be prepared to act upon new threats that may arise. Now this is not meant to scare you off, but just to make you think twice.

 

Why Manage your own E-Mail Server?


Why have your own e-mail server and manage it by yourself? Is that not why there are Internet Providers? Since you are, after all, reading this book you may already have your reasons, but let us anyway discuss these questions and some possible answers to them.The most important reason for hosting and managing your own e-mail server is control. For many organizations, e-mail is an important part of the IS/IT infrastructure. Keeping the control over your e-mail has many advantages:

  • If your company has offices in multiple places, you have full freedom when choosing how to connect them. Virtual private network between the offices, TLS-secured connections between the offices, a single server for all offices, one server per office, etc.

  • By keeping your own messaging in-house, you can send messages to each other without having them travel across the line to and from your provider. This gives a more reliable service if your Internet connection fails, and it avoids unnecessary latencies.

  • You are not dependent on the competence of the provider's staff. If you manage your own server and need to solve a difficult problem or implement a custom solution for something, just hire a consultant to help you.

  • If the provider goes bankrupt, all your data resides safely in your server room and on your backup media.

  • You are not subject to the limitations that your provider may set regarding, say, use of disk space or the maximum size of messages.

  • You can implement any policies for anti-spam or anti-virus that you choose.

More control requires more responsibility and more knowledge, and that is where this book comes in.

These hopefully compelling arguments aside, there are also downsides to hosting your own e-mail server. This is a task that requires a certain level of knowledge and commitment and so should not be undertaken be everyone. With your own server, you are not only responsible for the service you provide to your users, but you also have a responsibility towards the whole Internet community. An ill-configured e-mail server can help worms and spam to spread, which not only is a disservice to the community but can also get your server blacklisted. Even though a properly set up server can run for years without requiring much maintenance, you must keep yourself reasonably updated and be prepared to act upon new threats that may arise. Now this is not meant to scare you off, but just to make you think twice.

 

What you need to Host an E-Mail Server


Your server needs to be available through a permanent Internet connection with a fixed IP address. SMTP does provide an option to let you have your messages delivered to your provider's server, and then have those messages delivered to your own server upon request. The request can be automated and placed every time your connection is up. You can also retrieve new messages to your own server via the Post Office Protocol, (POP), normally used by end-user mail application. Neither of these solutions is especially good and they largely defeat the whole purpose of managing your own e-mail server.

In theory, it is possible to run an e-mail server with a non-fixed (dynamic) IP address, but it will not be reliable when the IP address is changed and you will risk losing messages. With a dynamic IP address, you will also face a bigger risk of being put on one of the blacklists for dynamic IP address ranges.

If you are serious about running an e-mail server, get a decent business-class Internet connection. These are inexpensive to come by these days, and investing in one may save you a lot of trouble later on. E-mail traffic does not depend on high bandwidth, so the capacity of a simple DSL line will do fine.

Even though you will need a fixed IP address, you do not necessarily need a public IP address dedicated to the mail server. If you only have a few external IP addresses and use private RFC 1918 addresses (192.168.x.y) on the inside with a Network Address Translation (NAT) router connecting your private network to the rest of the world, you can set up your router to forward the ports required by your e-mail services to your internal e-mail server. The table below shows which TCP ports are most likely to come in question for this:

Port

Service

25

SMTP

110

POP

143

IMAP

993

IMAP over TLS

If your employees want to access their messages from home or from the road, all you really have to do is make sure that no firewall is blocking access to the required ports and that your NAT router (if any) forwards these ports correctly. If your users want to send messages via your SMTP server, some extra configuration will be necessary to allow your server to authenticate to the SMTP server.

 

Sizing the Hardware of your E-Mail Server


When sizing a computer to use as an e-mail server, a lot of people have misconceptions regarding the hardware required to perform this task well. The constantly increasing performance of computers seems to lead people into thinking that they really need the latest and most buzzword-compliant stuff, even if they only want to handle a few thousand messages per day.

lthough a certain expertise is required to assess the hardware needs for an organization closely, common sense goes a long way. For a company with 100 users, a reasonably high upper limit for the number of messages per day would be 5,000. That would allow each user to send or receive 50 messages every day. Even if we say that each and every message is sent within the eight hours of the working day, on an average, the system will not have to cope with more than 10 messages per minute. Is it reasonable that a modern computer needs more than six seconds to receive and act upon a single e-mail message, often only a few kilobytes in size? No, it is not.

This little back-of-the-envelope exercise is obviously very rough and does not, for example, take into account the fact that messages typically do not arrive uniformly distributed in time, but it is still a pretty good way of estimating.

Let us now take a little deeper look into what to think about when sizing your server. For an SMTP server that does not perform any content scanning (viruses, spam, etc.), the performance is typically not bound by the CPU but by the I/O performance, specifically the seek time of the hard disk(s) and the quality and configuration of the I/O controller. Throwing more CPU horsepower at the problem will not help. Modern computers are relatively better equipped CPU-wise than I/O-wise, so investing in a multiple gigahertz dual-CPU configuration is probably useless. For any reasonably modern 1 GHz-class PC a handful of messages per second is no problem. That load equates to almost 20,000 messages every hour.

Adding content scanning will probably increase the CPU load quite a lot, and the I/O system will also require more power to keep up. Still, one or two messages per second should not place a noticeable load on the system.

What we have been discussing so far is just the SMTP server. All it does is receive messages and deliver them to other hosts or to local mailboxes. When sizing a server, do not forget that people are going to want to read their e-mail too. This service is provided by server software for POP, IMAP, or both. Just like SMTP software, the key requirement is I/O and not CPU. The number of users of the system is by itself an irrelevant figure; what are important are the usage patterns. How often will the users poll their mailboxes? If 100 users poll their mailboxes once every five minutes, on an average there will be one every three seconds. Checking if a mailbox has any new messages takes a fraction of a second, so the burden will not be significant.

These guidelines may appear vague and non-specific, but it is impossible to give exact figures. The performance you can get from a given piece of hardware depends on so many factors that trying to give anything but general guidelines would be misleading. Use common sense and simple back-of-the-envelope calculations; do not buy the fanciest server you can find unless you are sure you really need it, but also do not use any old abandoned desktop machine you can find. Even if the performance of the old desktop machine may suffice, the components may be old and the service agreement or warranty may be out of date.

 

Main E-Mail Protocols—SMTP, POP, and IMAP


Why are we discussing basic network communication protocols in this book? Are we not running advanced software? Indeed we are, but knowing one's way around the protocols can not only assist debugging a possibly non-working system but also increase the understanding of a mail system's behavior. We will start with a rather non-technical bird's-eye view of the protocols, after which we will dwell upon the protocol details.

Overview

In the UNIX environment, traditional mail applications have not used any network protocol at all. They have instead accessed the locally stored mailbox files directly through the file system. Typically, the inbox of each user is stored in a single file in either the /var/mail or the /var/spool/mail directory with the same name as that of the user (for example, /var/spool/mail/joe). The focus of this book is to discuss Linux-based e-mail solutions for the small office where users do not wish to log on to a central server with a terminal application in order to access their mail, so local mail storage will only be covered briefly on a need-to-know basis.

The most important protocol in Internet mailing is the Simple Mail Transfer Protocol (SMTP). Its simple purpose is to transport e-mail messages between two computers. These computers may either both be servers, or one of them may be a client machine on which the user runs the mail application—Outlook, Thunderbird, Eudora, or whatever. To collect new messages, the end user does not utilize SMTP. This is where the Post Office Protocol (POP), and the Internet Message Access Protocol, (IMAP), come in.

Some proprietary systems such as Microsoft Exchange and Lotus Notes use their own protocols to access messages, and we will not discuss them here.

POP Protocol

POP is the older and more widely used protocol of the two. It focuses on giving the users access to their inboxes, from which the users can download the new messages to their local computers and then delete them from the server. POP servers are not meant to be used for permanent storage of messages and the POP services of some Internet providers even prohibit users from leaving messages on the server after they have been downloaded once. The chief disadvantage of POP is that it only provides access to an intermediary storage medium and the users must store their messages permanently someplace else (for example, on their local hard drives). This is not only impractical for users who want to access their e-mail messages from multiple locations, but it is also a hassle for the system administrator who has to implement a backup solution for the users' messages. POP also does not have any notion of providing multiple folders for every user; with POP a user can access his or her inbox, period.

IMAP Protocol

IMAP is meant as an access method to a first-class mailstore, that is, it is designed to allow the user to store the messages permanently on the server. This solves the system administrator's backup problem and allows the user to access all messages from any place in the world (firewall restrictions aside). IMAP also has a more widespread implementation of TLS-secured connections, making IMAP safe to use in hostile environments. To improve performance and allow users to work with their mailboxes while not being connected to the mail server, most mail applications with IMAP support cache the downloaded mailboxes and messages in the local hard drive.

The SMTP Protocol

SMTP is a line-oriented text protocol that runs over TCP, which makes it trivial to decode SMTP transcripts and to initiate SMTP sessions using the regular Telnet client found on just about any computer. An SMTP client starts a session by connecting to port 25 on the SMTP server. After the server has greeted the client, the client must respond by saying hello, or actually HELO or EHLO, followed by the client's hostname. If the server accepts the cordial greeting, the client may begin the first mail transaction.

An SMTP mail transaction consists of three parts: a sender, one or more recipients, and the actual message contents. The sender is specified with the MAIL FROM command, each recipient with an RCPT TO command, and the start of the message contents with a DATA command. If the server accepts the message then the client may continue with additional transactions or issue the QUIT command to terminate the SMTP session.

Let us be less abstract and look at an actual SMTP session to give you a better idea. The bold face print represents what the client sends to the server.

220 mail.example.com ESMTP Postfix (2.2.3)
EHLO gw.example.net
250-mail.example.com
250-PIPELINING
250-SIZE
250-VRFY
250-ETRN
250 8BITMIME
MAIL FROM:<[email protected]> SIZE=112
250 Ok
RCPT TO:<[email protected]>
250 Ok
RCPT TO:<[email protected]>
250 Ok
RCPT TO:<[email protected]>

550 <[email protected]>: Recipient address rejected: User unknown in local recipient table

DATA
354 End data with <CR><LF>.<CR><LF>

Subject: Test mailTo: <[email protected]>Date: Sun, 15 May 2005 20:23:22 +0200 (CEST)This 
is a test message..

250 Ok: queued as B059D3C2B

QUIT
221 Bye

This example shows a host that claims to be named gw.example.net connects to an SMTP server that calls itself mail.example.com. Because the server's first response contains ESMTP, the client decides to try Enhanced SMTP (ESMTP) and greets the server with EHLO instead of HELO. The server accepts this greeting and responds with a list of the supported ESMTP extensions.

Together with the sender address, the client sends the SIZE attribute to indicate the size of the message to the server. This is allowed because the server has stated that it supports the SIZE extension. If the size specified by the client exceeds the message size limit set by the server, the message can be rejected at once rather than after the whole message has been received and the server can assess the size.

An SMTP message can obviously have more than one recipient. This has a few consequences that must be remembered while implementing a mail system and inventing policies. In the example above, the mail server accepts the first two recipients but rejects the third one. Since two recipients have been accepted by the server, the client will try to send the message contents. Here the message is accepted by the server and queued for delivery (250 Ok: queued as B059D3C2B), which means that the SMTP server has taken over the responsibility for the delivery of the message to the accepted recipients. If the message cannot be delivered, the server will send a non-delivery message (bounce) back to the sender. The server could also have chosen to reject the whole message. If so, it would have rejected it for all recipients and not delivered it at all. In other words, in response to the message contents the server must either reject the message for all recipients or accept it for all recipients.

It is vital to understand the difference between the envelope and the header. The envelope of a message consists of the information given in the MAIL FROM and RCPT TO commands, that is, the sender and recipient information that is used to deliver the message. An SMTP server pays no attention what so ever to the From, To, and Cc message headers. In our example the To header contains just a single address with no other relation to the actual recipient addresses than the domain, but that is just a coincidence. Bounces are always sent to the envelope sender address, in this case [email protected]. The sender address of bounce messages is the empty sender address, often called the null sender. However tempting it may be for some people, the null sender address must not be blocked.

So far we have not commented on the numerical codes given by the server at the beginning of each line. Each number has a specific meaning and it is important to learn the correct interpretation of the first digit.

Digit

Meaning

2

Server has accepted the previous command and is awaiting your next command.

3

Used only in response to the DATA command, and means that the server is ready to accept the message contents.

4

Temporary error: the request cannot be performed at the moment, but it may be successfully serviced later.

5

Permanent error: the request will never be accepted.

In SMTP, error conditions can be either temporary or permanent. Both 4 and 5 are used to signal errors. A client that receives a temporary error designated by 4 should disconnect, keep the message in the queue, and retry at a later time. Typical temporary error conditions include a full mail queue disk, a server configuration error that must be resolved before messages can be accepted, or a temporary DNS lookup error. Permanent errors are indicated by the first digit being 5 and mean that the request will never be accepted, so a client will have to remove the message from the queue and send a bounce to the sender telling him or her that the message could not be delivered.

There is a lot more to SMTP than this quick introduction has covered, but this should get your started. A number of RFC documents cover SMTP related topics, but the probably most important ones are RFC 821 (Simple Mail Transfer Protocol) and RFC 822 (Standard for the format of ARPA Internet text messages).

 

E-Mail and DNS


The Domain Name System, or DNS, plays an important role in e-mailing. The DNS is used by both e-mail clients and e-mail servers. Even if you do not intend to maintain your own DNS server, a thorough understanding of DNS's role in e-mailing is a necessity for the mail server operator. This section assumes that the reader has basic knowledge of how DNS works in general.

DNS Record Types Used by E-Mail Applications

In many networking scenarios, only two DNS record types are used: A and PTR. These map hostnames to IP addresses and IP addresses to hostnames, respectively. These record types are also used for e-mail, but there is also a third DNS record type that is uniquely available for e-mail.

How does an SMTP server discover to which host a message for a certain domain should be delivered? The recipient domain is, not surprisingly, used as the key in one or more DNS lookups. The first lookup that is made is for the mail-specific MX record—the mail exchanger record type that allows the DNS operator to specify the hostname or hostnames of servers that receive mail for a certain domain. For example, MX records can be used to specify that messages to someone at example.com should be sent to mail.example.com. If the recipient domain does not have an MX record, an attempt is made to find an A record for the recipient domain. If the A record lookup succeeds, the mail will be delivered to the host. If both the MX and A lookups do not return any results, the message is deemed undeliverable and is returned to the sender.

What is the point of having these MX records? There are two good reasons:

First, it might not be desirable to be forced to map the A record of a domain to the mail server. What if Example, Inc. with the WWW address http://www.example.com/ wants to allow visitors to use the shorter http://example.com/ URL, but does not want to run the web server application on the mail server (or vice versa)?

The more important reason is that the result of an MX lookup does not only contain a list of hostnames, but rather a list of (hostname, priority) tuples. The priority field is an integer describing the priority of the hostname within the list. The absolute magnitude of the priority number does not matter, but it is used in relation to the priority of any other hostnames to create an ordered list of hostnames to try when delivering a message. The list is in ascending order, so the hostname with the lowest priority number will be contacted first. If two hostnames have equal priority, they will be tried in random order.

Equal-priority MX records can be used as a very crude form of load balancing between two or more servers, but this is also possible with A records that map to multiple IP addresses. The hierarchy of backup mail servers with different priorities that can be set up for a domain using MX records cannot be made to happen with A records. Let us look at a constructed example of an organization that uses a lot of mail servers:

Priority

Hostname

10

mx1.example.com

10

mx2.example.com

20

mx3.example.com

30

mx4.example.com

If this DNS configuration is set for the domain example.com, SMTP servers are expected to try to deliver messages for example.comto mx1.example.comto mx1.example.com or mx2.example.com first. If both connections fail, mx3.example.com should be tried, and if even that server does not respond in a timely way, mx4.example.com is the last resort. Should that fail too, the message is kept and delivery is retried at a later time.

 

Backup Mail Servers


Having a backup server that can receive your messages if your own server is unavailable sounds like a really good idea, but today's reliable Internet connections together with spam, worms, and other rubbish have for the most part made backup mail servers unnecessary and often even harmful. The rationale for having a backup server is that it can receive messages while your primary server is down, and then deliver them to the primary server when it is up again. The advantage of this is, however, very small since all SMTP servers are required to queue undeliverable messages anyway for at least five days before they are returned to the sender. Granted, by having a backup server you can choose to store unavailable messages for longer time than five days, but if your main SMTP server is unavailable longer than five days at a stretch you probably have bigger problems than a few lost messages.

But how does the spam fit in? Because a backup mail server typically does not have the same spam-thwarting configuration as the primary server, spammers often specifically target backup servers in order to bypass the stricter rules of the primary server.

Another strong reason to avoid backup mail servers is that they typically do not perform recipient validation. This means that they do not know which recipient addresses are valid for the domains they act as a backup servers for. This requires a backup server to accept all messages for the backed-up domains and attempt to deliver them to the primary server. The primary server will reject invalid recipients, causing the backup server to bounce such message back to the sender. This is known as backscatter and is bad for two reasons; sender addresses are often spoofed, so the bounces may be sent to an innocent bystander, and it may fill your mail queue with bounced messages that cannot be delivered because the receiving server is unavailable. A busy server that does not perform recipient validation and is hit heavily with spam may have thousands or tens of thousands of undeliverable messages residing in the queue.

 

Summary


In this chapter, we started by discussing why you should even consider hosting your own e-mail server. Then, we looked at some questions that need to be answered before you can start working with your server—the kind of network connection and computer power you can expect to need for your server. To manage an e-mail server successfully, an understanding of the network communication protocols used is important. We gave an overview of POP and IMAP, and delved more deeply into the most important of them all, SMTP. Finally, we looked at the vital role that the DNS plays in e-mailing.

About the Authors

  • Alistair McDonald

    Alistair McDonald is a freelance IT consultant based in the UK. He has worked in IT for over 15 years and specializes in C++ and Perl development and IT infrastructure management. He is a strong advocate of open source, and has strong cross-platform skills. He prefers vim over vi, emacs over Xemacs or vim, and bash over ksh or csh.

    He is very much a family man and spends as much time as possible with his family enjoying life.

    Browse publications by this author
  • Carl Taylor

    Carl Taylor has worked over 20 years in the IT industry and has spent the majority of that time working on Unix type systems, mainly communications or office automation projects. He was an early user of the UseNet network and taught himself to programme in C through working on a variety of open source software. His experience covers roles including pre and post sales support, product development, end user training and management.

    Carl now runs his own Web Solutions development company 'Adepteo' where they specialise in intranet and workflow products building on the best open source applications available. Whilst not working or looking after his children Carl is something of a dance addict and is currently learning Latin and Ballroom and Salsa.

    Browse publications by this author
  • David Rusenko

    David Rusenko was born in Paris, France, and spent most of his childhood overseas. He began working as a freelance web designer in 1996 and had his first experience with open source, a box copy of RedHat 5.2, shortly after in 1999. After six years and as many versions of RedHat, he now creates appealing web pages and devises solutions implementing high availability through clustering and alternate security models.

    He founded Aderes in 2001, a company which provides email and web-based security solutions. His search for an appropriate Webmail platform for the company led him to Squirrelmail. Initially managing all aspects of the business, from the technical concerns to customer support, gave him the experience he now contributes to the Webmail chapter of this book.

    David has studied both Information Sciences and Technology (IST) and Management Information Systems (MIS) at the Pennsylvania State University. He speaks English and French fluently, and is conversational in Arabic. During his free time and vacations, he enjoys scuba diving, backpacking, playing racquetball and playing electronic music records.

    Browse publications by this author
  • Magnus Back

    Magnus Back has been playing and working with computers since he was a kid, and within the computer field he is interested in everything from digital typography and compilers to relational databases and Unix. His interests also include e-mail services, and he is an active contributor to the Postfix mailinglist.

    Magnus holds a master's degree in computer science and engineering from Lund Institute of Technology, Sweden, and currently works with software configuration management and tools development for GSM/UMTS phones at Sony Ericsson Mobile Communications.

    Browse publications by this author
  • Patrick Ben Koetter

    Patrick Ben Koetter is an active and well-known figure in the Postfix community, working as information architect. Patrick Koetter runs his own company consulting and developing corporate communication for customers in Europe and Africa.

    He speaks about Postfix at industry conferences and hacker conventions and contributes regularly to a number of open source mailing lists. Patrick Koetter is Co-author of 'The Book of Postfix'.

    Browse publications by this author
  • Ralf Hildebrandt

    Ralf Hildebrandt is an active and well-known figure in the Postfix community, working as a systems engineer for T-Systems, a German telecommunications company.

    He speaks about Postfix at industry conferences and hacker conventions and contributes regularly to a number of open source mailing lists. Ralf Hildebrandt is Co-author of 'The Book of Postfix'.

    Browse publications by this author
Book Title
Access this book, plus 7,500 other titles for FREE
Access now