Remote SPAM Filtering Using Spamassassin

Why Remote Filtering?

SMTP servers serving a large user base are usually too busy for their performance to be sacrificed to filter out messages sent by idiot spammers. (Yes, Spammers, go ahead and sue me! You are idiots and you're a nuisance like mosquitos. All spammers must be exterminated!) Besides, there are people (believe it or not) who won't like their mail to be filtered against spam. I found out by experience that, spam filtering has to be optional; filtering out spam at the mail gateway causes unnecessary arguments with the users.

Thanks to UNIX (and of course to LINUX) everything is configurable... The extremely handy "procmail" helped us solve the "optional remote spam filtering" problem.

Tools

Spamassassin is a very good open source tool to filter out SPAM. Installation is easy (assuming you're using a UNIX or LINUX mail server; otherwise you should be used to doing things the hard and expensive way), configuration is easy, tailoring is easy and better still, it is trainable against new spamming techniques devised by the innovative hurd of bad guys. procmail is the next important tool we make use of (procmail is a standard mail delivery tool on almost all UNIX flavors, su you shouldn't worry about its existence on your system). And, finally, PERL; as usual (thanks Larry, for the wonderful tool).

How it works?

Users who wish their incoming mail to be filtered against spam need to forward their mail to a predefined mail alias on a specially configured server. On our system, the alias is "filter@nospam.bilkent.edu.tr". The host nospam.bilkent.edu.tr is a typical LINUX box with the following important characteristics:
The filtering mechanism is simple:  a script running at the address filter@nospam.bilkent.edu.tr ( can forward incoming mail to a program; details will soon follow) receives the incoming mail, checks it using spamassassin. Spamassassin marks the e-mail as SPAM if it believes so and the .forwardfilter.pl script sends it back to the original user to be placed in the mailbox. Graphically...

route

Details (Users' perspective)

To forward an incoming e-mail to another host, where it will be checked against SPAM, we make use of the fantastic procmail. A user who wants his/her e-mail to be SPAM checked must create two dot files in his/her home directory; namely the .forward and .procmailrc  files. Remember, this works ONLY on UNIX and LINUX mail servers. Pity that there is no procmail for Windows variants.

The .forward  file should contain:

    "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #user"

Please note that every single byte in the above line is significant, including the double quotes.

The .procmailrc  file, on the other hand, should contain:

:0
* ^X-Filter
$DEFAULT

:0
* < 256000
! filter@nospam.bilkent.edu.tr

:0
$DEFAULT

Now, now... What's happening here...
First things first; again every single byte in the above lines are significant. SÝte specific values are in red :

The first 3 lines comprise the first rule.  It reads "If the headers of an incoming mail contain a line starting with "X-Filter", deliver it to the default mailbox. (A X-Filter header means that the e-mail has been scanned by spamassassin and gone through filter.pl).

The second group of 3 lines is the second rule. The number 256000 is the size limit for mail to be checked against spam. I've assumed that spammers won't send out spam mail larger than 256KBytes. Afterall, sending out large mails to hundreds of thousand of people doesn't make much sense from their point of view. You certainly can increase or decrease this size limit. The rule redirects e-mails with total size les than 256000 bytes to an e-mail address on the remote filtering host.

The line which starts with an exclamation mark and contains "filter@nospam.bilkent.du.tr" is an example. This is a specially configured e-mail address on the remote spam checker host. In other words, in the above example, nospam.bilkent.edu.tr is the host on which spam checking software (namely spamassassin) and a check-forward application is installed. "filter" is a specially configured mail alias setup on this host.

The third group of 2 lines if the "otherwise" rule which causes unchecked mail with size greater than 256000 bytes to be delivered to the user's default mailbox.

Now, we know that most of the users on your network will have problems in creating these two dot files correctly. Therefore we have also written two small shell scripts to create and remove these two files without asking any questions (see below).

Please note that these two files can only be created on "end-point" email accounts. You shouldn't try to modify the .forward file for an account whose e-mail are forwarded to another address using a .forward file. This scheme works only for "final" accounts; i.e. the final recipient addresses on forward chains. Please also note that .forward and .procmail files won't work if they are not owned by the account owner and/or writable by others (a very clever UNIX security measure). Therefore these files must be set with
   
    chown cayfer .forward .procmailrc
    chmod 600 .forward .procmailrc

commands as well.

Details (sysadmins' perspective)

The sysadmin must have a separate UNIX/LINUX box with spamassassin installed and running as a daemon. In my case, this is a separate host named "spamdnospam.bilkent.edu.tr". I have spamd installed and running ( /etc/rc.d/init.d/spamd start ). I also occasionally train my spamassassin with sample spam mails.

Then I have a mail alias called "filter" which is created with the following line in my /etc/aliases file. (I use postfix as the SMTP server and it is configured to use hashed alias tables.)

filter: "| /usr/bin/spamc -t 10 | /usr/share/spamassassin/filter.pl"

Again, every single byte is significant. What this alias does is: It redirects all incoming mail to the command

/usr/bin/spamc -t 10 | /usr/share/spamassassin/filter.pl

which runs the spamassassin client (spamc) with a timeout of 10 seconds and redirects its output to our filter.pl Perl script installed under /usr/share/spamassassin.

I've configured spamassassin so that it prefixes the subject of a spam message with "***SPAM***". Furthermore, if spamassassin decides that an email is SPAM, it inserts a header which reads:  "X-Spam-Flag: YES". The Perl script "filter.pl" parses the output of the "spamc" command; finds out the real recipient and sender of the message; reorgani[sz]es the headers (prepends "X-" to most header lines so that the  SMTP server doesn't detect a mail loop) and sends its back to the original recipient.

The filter.pl program

This is a simple Perl script which parses the output of spamc  (the spamassassin client) to find who the sender and recipient of the original message were; modify the original headers by prepending "X-" and resending the spamc output the original recipient. Note that modifying the original headers is crucial; otherwise SMTP servers will detect email loops; i.e. messages bouncing between two addresses indefinitely.

Like all well behaving UNIX programs, filter.pl logs its actions to syslog. You can  see what this program has logged using the command:

grep  SPAM-FILTER  /var/log/syslog

Installing filter.pl

This step is very easy. Just download filter.pl; save and move it to /usr/share/spamassassin/filter.pl; set it world executable and owned by root. (A more detailed install procedure is described in the script itself.)

chown root:root /usr/share/spamassassin/filter.pl
chmod a+x /usr/share/spamassassin/filter.pl

Helping your users setting up their  .forward and .procmailrc files

We use two simple shell scripts that we have installed in /usr/local/bin and we tell our users (who want to have their mail checked against spam) to login to their regular accounts and execute the "start_filtering" command. This shell script checks whether they already have any .forward or .procmailrc files in their homes; and creates both files if it is appropriate. They can issue the "stop_filtering" command anytime they want to resume learning more about enlargement of various parts of the human body.

#!/usr/bin/sh
###################
# start_filtering #
###################
# This script is a supplement of Remote-SPAM-Filtering
#
# Setting up Remote SPAM Filtering requires the users
# to place two mail delivery related dot files in their
# home directories.
#
# These files are .forward and .procmailrc
#
# by: Can Ugur Ayfer - Bilkent University
#     May 2004
######################################################
# There should be no .forward .procmailrc files
# in the home dir
#

if test -f "$HOME/.forward"; then
   echo " "
   echo "You already have a .forward file in your home directory."
   echo "The .forward file contains the line(s):"
   echo " "
   cat $HOME/.forward
   echo " "
   echo "We CANNOT enable spam filtering for this account."
   echo " "
   exit 1
fi

if test -f "$HOME/.procmailrc"; then
   echo " "
   echo "You already have a .procmailrc file in your home directory."
   echo "The .procmailrc file contains the line(s):"
   echo " "
   cat $HOME/.procmailrc
   echo " "
   echo "We CANNOT enable spam filtering for this account."
   echo " "
   exit 1
fi

#
# create .forward and .procmailrc files
cat > $HOME/.forward << __EOF__
"|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #user"
__EOF__

chmod 600 $HOME/.forward

cat > $HOME/.procmailrc << __EOF__
:0
* ^X-Filter.*Bilkent
\$DEFAULT

:0
* < 256000
! filter@nospam.bilkent.edu.tr

:0
\$DEFAULT
__EOF__

chmod 600 $HOME/.procmailrc

echo " "
echo "   SPAM checking is now ENABLED for your account. "
echo " "
echo "   Please send yourself an e-mail to make sure that "
echo "   the setup is working fine."
echo "   If you believe that there is a problem"
echo "   with your incoming mail, please issue"
echo "   the stop_filtering command to undo the changes."
echo " "
echo "   Any mail that the scanner software believes"
echo "   to be a spam, will be marked with *****SPAM*****"
echo "   in the subject line and sent as a regular"
echo "   mail to your mailbox. "
echo " "
echo "   If you believe the spam scanner works fine,"
echo "   you can set your e-mail program's filters"
echo "   so that all marked messages are sent to Trash"
echo "   automatically."
echo " "





#!/usr/bin/sh
###################
# stop_filtering  #
###################
#
# This script is a supplement of Remote-SPAM-Filtering
#
# Disabling Remote SPAM Filtering requires the users
# to remove the two mail delivery related dot files in their
# home directories.
#
# These files are .forward and .procmailrc
#
# by: Can Ugur Ayfer - Bilkent University
#     May 2004
######################################################
#
# There should be a .forward and a .procmailrc file in the home dir

if  [ -f $HOME/.forward -a -f $HOME/.procmailrc ]
then
   echo " "
   echo ".forward file contains:"
   cat $HOME/.forward
   echo " "
   echo " "
   echo ".procmailrc file contains:"
   cat $HOME/.procmailrc

   echo " "
   echo "Are you sure that you want to delete these two files? (y/N): "
   read ANS
   case $ANS in
     [yY])
         /bin/rm $HOME/.procmailrc
         /bin/rm $HOME/.forward
         echo " "
         echo "Your .procmailrc and .forward files are now deleted."
         echo "Spam checking is now disabled for your account."
         echo " "
         echo "You can enable spam checking by issuing the command \"start_filtering\""
         echo " "
         exit;;
   esac
   echo " "
   echo "Nothing changed."
   echo " "
   exit
fi

echo " "
echo "     You do not have a .forward or .procmailrc file!"
echo "     This means your account is NOT set for spam checking."
echo "     There is no sense trying to stop spam checking for this account!"



 

May 2004