Linux E-mail: Using SpamAssassin

by Alistair McDonald | November 2009 | Linux Servers Open Source

In this article series by Alistair McDonald, we will cover some important topics that discuss the use of Spam Assassin in conjunction with Procmail to filter out the wide range of spam that afflicts the modern e-mail user.

In this article, we will learn:

  • How to filter incoming e-mails with SpamAssassin?
  • How to configure SpamAssassin to work on per-user or per-server basis?

Now that SpamAssassin is installed, we need to configure the system to use it. SpamAssassin can be used in many ways. It can be integrated into the MTA for maximum performance; it can run as a daemon or a simple script to avoid complexity; it can use separate settings for each user or use a single set of settings for all users; and it can be used for all accounts or just for the chosen ones. In this article, we will discuss using SpamAssassin in three ways.

The first method is with Procmail. This is the simplest method to configure and is suitable for low-volume sites, for example, less than 10,000 e-mails a day.

The second method is to use SpamAssassin as a daemon. This is more efficient, and can still be used with Procmail, if desired.

The third method is to integrate SpamAssassin with a content filter such as amavisd. This offers performance advantages, but occasionally the content filter does not work with the latest release of SpamAssassin. Problems, if any, are usually resolved quickly.

To help you get the most out of SpamAssassin, Packt Publishing has published SpamAssassin: A practical guide to integration and configuration, (ISBN 1-904811-12-4) by Alistair McDonald.

Using SpamAssassin with Procmail

Before we configure the system to use SpamAssassin, let's consider what SpamAssassin does. SpamAssassin is not an e-mail filter. A filter is something that changes the destination of an e-mail. SpamAssassin adds e-mail headers to an e-mail message to indicate if it is spam or not.

Consider an e-mail with headers like this:

Return-Path: <user@domain.com>
X-Original-To: jdoe@localhost
Delivered-To: jdoe@host.domain.com
Received: from localhost (localhost [127.0.0.1])
by domain.com (Postfix) with ESMTP id 52A2CF2948
for <jdoe@localhost>; Thu, 11 Nov 2004 03:39:42 +0000 (GMT)
Received: from pop.ntlworld.com [62.253.162.50]
by localhost with POP3 (fetchmail-6.2.5)
for jdoe@localhost (single-drop); Thu, 11 Nov 2004 03:39:42 +0000
(GMT)
Message-ID: <D8F7B41C.4DDAFE7@anotherdomain.com>
Date: Wed, 10 Nov 2004 17:54:14 -0800
From: "stephen mellors" <gregory@anotherdomain.com>
User-Agent: MIME-tools 5.503 (Entity 5.501)
X-Accept-Language: en-us
MIME-Version: 1.0
To: "Jane Doe" <jdoe@domain.com>
Subject: nearest pharmacy online
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit

SpamAssassin will add header lines.

X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 3.1.0-r54722 (2004-10-13) on
host.domain.com
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.8 required=5.0 tests=BAYES_05,HTML_00_10,
HTML_MESSAGE,MPART_ALT_DIFF autolearn=no
version=3.1.0-r54722

SpamAssassin doesn't change the destination of the e-mail, all it does is add headers that enable something else to change the destination of the e-mail.

The best indication that an e-mail is spam is the X-Spam-Flag. If this is YES, SpamAssassin considers the mail to be spam and it can be filtered by Procmail.

SpamAssassin also assigns a score to each e-mail—the higher the score, the more likely that the e-mail is spam. The threshold that determines if an e-mail is spam can be configured on a system-wide or per-user basis. The default of 5.0 is a sensible default if you are using an unmodified installation of SpamAssassin without any custom rulesets.

Global procmailrc file

Let's suppose that we want to check all incoming e-mail for spam using SpamAssassin. Commands in the /etc/procmailrc file are run for all users, so executing SpamAssassin here is ideal.

The following simple recipe will run SpamAssassin for all users when placed in /etc/procmailrc:

:0fw
| /usr/bin/spamassassin

To place all spam in an individual spam folder, ensure that the global/etc/procmailrc file has a line specifying a default destination. For example:

DEFAULT=$HOME/.maildir/

If not, then add a line specifying DEFAULT. To filter spam into a folder, add a recipe similar to the following:

* ^X-Spam-Flag: Yes
.SPAM/new

This assumes that each user has a folder called SPAM already configured.

To place all the spam in a single, central folder, use an absolute path to the destination in the recipe:

* ^X-Spam-Flag: Yes
/var/spool/poss_spam

This will place all spam in a single folder, which can be reviewed by the system administrator. As times, regular e-mail may occasionally be wrongly detected as spam, the folder should not be world-readable, which leads to a more generalized statement.

SpamAssassin will be run under the system account used by Postfix. This means that the Bayesian database and the auto-whitelists and blacklists will be shared by all users. From a security point of view, it's important that the various databases that SpamAssassin creates are not world-writable.

SpamAssassin stores user-specific files in the ~/.spamassassin/ directory. Here is a list of fi les that may be present for a user:

//===INSERT TABLE 01===

Some of them may contain confidential data—for example, regular contacts will appear in the auto-whitelist files. Careful use of permissions will ensure that the files are not readable by regular user accounts.

Using SpamAssassin on a per-user basis

Perhaps some users don't receive spam, or there may be issues with users sharing whitelists and Bayesian databases. SpamAssassin can be run on an individual basis by moving the recipes to the ~/.procmailrc of specific users. This should increase the filtering performance for each user, but increases disk space usage for each user and requires setting up each individual user account by modifying its ~/.procmailrc.

A typical user's .procmailrc might look like this:

MAILDIR=$HOME/.maildir
:0fw
| /usr/bin/spamassassin
:0
* ^X-Spam-Flag: Yes
.SPAM/cur

As suggested, e-mail may sometimes be wrongly detected as spam. It's worthwhile reviewing spam to ensure that legitimate e-mails have not been wrongly classified. If the user receives a lot of spam, then wading through it all is time consuming, tedious, and error prone. Procmail can filter spam by checking the spam score written in the e-mail headers by SpamAssassin.

The low-scoring spam (for example, scoring up to 9) can be placed in one folder called Probable_Spam, while higher scoring e-mails (which are more likely to be spam) can be placed a folder called Certain_Spam.

To do this, we use the X-Spam-Level header, which SpamAssassin creates. This is simply the number of asterisks, related to the X-Spam-Level value. By moving e-mail with more than a certain number of asterisks to the Certain_Spam folder, the remaining spam is "Probable Spam". E-mail that is marked with X-Spam-Flag: NO, is obviously not spam.

The following .procmailrc file will filter high scoring spam separately from low scoring spam and non spam:

MAILDIR=$HOME/.maildir
:0fw
| /usr/bin/spamassassin
:0
* ^X-Spam-Level: **************
.Certain_Spam/cur
:0
* ^X-Spam-FLAG: YES
.Probable_Spam/cur

Using SpamAssassin as a daemon with Postfix

A daemon is a background process; one that waits for work, processes it, and then waits for more work. Using this approach actually improves performance (as long as there is sufficient memory) because responsiveness is improved—the program is always ready and waiting and does not have to be loaded each time spam tagging is required.

To use SpamAssassin as a daemon, a user account should be added—it's dangerous to run any service as root. As root, enter the following commands to make a user and a group called spam:

# groupadd spam
# useradd -m -d /home/spam -g spam -s /bin/false spam
# chmod 0700 /home/spam

To configure Postfix to run SpamAssassin, use SpamAssassin as a daemon. The Postfix master.cf file must be changed. Edit the file and locate the line that begins with 'smtp inet'. Amend the line to add -o content_filter=spamd to the end.

smtp inet n - n - - smtpd -o content_
filter=spamd

Add the following lines to the end of the file:

spamd unix - n n - - pipe
flags=R user=spam argv=/usr/bin/spamc
-e /usr/sbin/sendmail -oi -f ${sender} ${recipient}

If the text is spread across several lines, any continuing line must begin with spaces as shown. The changes to the file define a filter called spamd that runs the spamc client for each message and also specifies that the filter should be run whenever an e-mail is received via SMTP.

On this line, spamd is the name of the filter and matches the name used in the content_filter line. The user= portion specifies the user context that should be used to run the command. The argv= portion describes the program that should be run. The other flags are used by Procmail and their presence is important.

Using SpamAssassin with amavisd-new

amavisd-new is an interface between MTAs and content checkers. Despite its name, amavisd-new is a well-established open source package that is well maintained. Content checkers scan e-mail for viruses and/or spam. amavisd-new is slightly different. Just like like spamd, it is written in Perl and runs as a daemon, but instead of accessing SpamAssassin via the spamc or spamassassin clients, it loads SpamAssassin into memory and accesses the SpamAssassin functions directly. It is therefore closely coupled to SpamAssassin and may need to be upgraded at the same time as SpamAssassin.

Unlike other Perl-based applications and utilities, amavisd-new is not available from CPAN. However, it is available in source form and RPM form for many distributions of Linux, and is also available for debian-based repositories. Details of versions available are listed on http://www.ijs.si/software/amavisd/#download. We recommend that if the version of SpamAssassin that your distributor offers is up-to-date, then you should use their package of both SpamAssassin and amavisd.

Installing amavisd-new from package

To install amavisd-new from package, use the rpm command for RPM-based distributions. amavisd-new has many dependencies, all of which are Perl modules. Each version may have different dependencies, which are listed in the install file that is a part of the package. The Perl prerequisites for version 2.6.2 are as follows:

Archive::Zip
BerkeleyDB
Convert::BinHex
Convert::TNEF
Convert::UUlib
Crypt::OpenSSL::Bignum
Crypt::OpenSSL::RSA
Digest::HMAC
Digest::Sha1
IO::Multiplex
IO::Stringy
MIME::Tools
Mail::DKIM
Net::CIDR
Net::DNS
Net::IP
Net::Server
Unix::Syslog

To view the prerequisites for a particular version of amavisd-new, download the source and unpack it as shown here, and read the install file.

$ cd /some/dir
$ wget http://www.ijs.si/software/amavisd/amavisd-new-2.6.2.tar.gz
$ tar xfz amavisd-new-2.6.2.tar.gz
$ cd amavisd-new-2.6.2
$ vi INSTALL

Several of the dependencies may be installed already, as they are also used by SpamAssassin.

Installation prerequisites

Some RPM-based Linux distributions may automatically install the prerequisites as dependencies. For other distributions, all the prerequisites must be downloaded from CPAN and installed. This is easiest to accomplish with the cpan command. An alternative method is to download the source code for each prerequisite individually and install it with the following commands:

$ cd /some/directory
$ gunzip -c source-nn.tar.gz | tar xf -
$ cd source-nn
$ perl Makefile.pl
$ make test
$ su
# make install

Installing from source

amavisd-new has no makefile, configuration script, or installation routine. To install it, the sole executable script is copied to /usr/local/bin, and its attributes modified to ensure it cannot be modified by non-root users:

# cp amavisd /usr/local/sbin/
# chown root /usr/local/sbin/amavisd
# chmod 755 /usr/local/sbin/amavisd

The sample amavisd.conf file should be copied to /etc and its attributes should also be modified.

# cp amavisd.conf /etc/
# chown root /etc/amavisd.conf
# chmod 644 /etc/amavisd.conf

amavisd-new must be configured to run as a daemon, and so the sample init script should be copied to the appropriate directory.

# cp amavisd_init.sh /etc/init.d/amavisd-new

The init script should also be added to the system startup. Most Linux distributions use the chkconfig command to do this.

# chkconfig --add amavisd-new

Creating a user account for amavisd-new

To create a user account, first create a dedicated group using the groupadd command and then use the useradd command to add the user.

# groupadd amavis
# useradd -m -d /home/amavis -g amavis -s /bin/false amavis

Configuring amavisd-new

Several changes need to be made to the /etc/amavisd.conf file. This file will be parsed as Perl source, and syntax is important. Each line should end in a semicolon, and the casing is important. The following variable declaration lines should be changed to contain the following values:

$MYHOME = '/home/amavis';
$mydomain = 'domain.com';
$daemon_user = 'amavis';
$daemon_group = 'amavis';
$max_servers = 5; # number of pre-forked children
(default 2)

Ensure that the correct domain is specified for $mydomain. The number 5 specified for $max_servers is the number of daemons that will be run concurrently. If you have a modest amount of e-mail, for example less than ten messages a second, the default will be sufficient.

Within /etc/amavisd.conf, there is a section on SpamAssassin-related configuration settings:

$sa_tag_level_deflt = 2.0;
$sa_tag2_level_deflt = 6.2;
$sa_kill_level_deflt = 6.9;

These three settings are used with the SpamAssassin score level associated with the e-mail being processed. The $sa_tag_level_deflt setting is the threshold at which ham is separated from spam and the X-Spam-Status and X-Spam-Level headers are added to an e-mail.

E-mails that score below this threshold do not have headers added, while e-mails above the threshold will have headers added. The $sa_kill_level_deflt setting is the threshold at which spam e-mail is rejected.

The default configuration is to reject spam. To forward spam to another e-mail address, locate the line specifying $final_spam_destiny or add one if it is not present, and make it read as follows:

$final_spam_destiny = D_PASS; # (defaults to D_REJECT)

The recipient of the spam has to be defined. Locate the line that specifies $spam_quarantine_to, and alter it or add one to contain an e-mail address. The $mydomain variable, which was configured earlier in this step, can be used to refer to the domain—remember to prefix the @ symbol with a backslash.

$spam_quarantine_to = "spam-quarantine@$mydomain";

Now, amavisd-new should be started. Most Linux distributions use the following command:

# /etc/init.d/amavisd-new start

Configuring Postfix to run amavisd-new

Edit /etc/postfix/master.cf and locate this line:

smtp inet n - n - - smtpd

Add these lines after it:

smtp-amavis unix y - 5 smtp
-o smtp_data_done_timeout=1200
-o disable_dns_lookups=yes
127.0.0.1:10025 inet n y-- smtpd
-o content_filter=
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes

In the smtp-amavis line, the number 5 specifies the number of instances that can be used at once. This should correspond to the $max_servers entry specified in the amavisd.conf file.

Edit /etc/postfix/main.cf and add the following line near the end of the file:

content_filter = smtp-amavis:[localhost]:10024

Restart Postfix with the postfix reload command:

# postfix reload

Summary

In this article, we have learnt:

  1. How to filter incoming e-mails with SpamAssassin?
  2. How to configure SpamAssassin to work on per-user or per-server basis?

If you have read this article you may be interested to view :


About the Author :


Books From Packt

ModSecurity 2.5
ModSecurity 2.5

FreePBX 2.5 Powerful Telephony Solutions
FreePBX 2.5 Powerful Telephony Solutions

Asterisk 1.4 – the Professional’s Guide
Asterisk 1.4 – the Professional’s Guide

trixbox CE 2.6
trixbox CE 2.6

Cacti 0.8 Network Monitoring
Cacti 0.8 Network Monitoring

Tomcat 6 Developer's Guide
Tomcat 6 Developer's Guide

Apache Maven 2 Effective Implementation
Apache Maven 2 Effective Implementation

Drools JBoss Rules 5.0 Developer's Guide
Drools JBoss Rules 5.0 Developer's Guide

No votes yet

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
P
U
G
5
6
g
Enter the code without spaces and pay attention to upper/lower case.
Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software