Reduce SPAM and improve security – Amavis + SpamAssassin + ClamAV + Procmail + PostScreen

http://upload.wikimedia.org/wikipedia/commons/b/b7/SpamAssassin_logo.png>90% of mail traffic are actually SPAM….and you will quickly need to implement Spam protection either from global blacklist, or learning algorithm or even check SMTP protocol respect.

The most popular way to block SPAM on your mail server is probably SpamAssassin. It’s a free and Open Source spam filter written in Perl. It will perform a wide range of tests on headers and body text to determine how likely spam will be your mail. You could after make SpamAssassin learn from its mistake (Ham) or endorse its correct decision (SPAM). It’s a powerful too and very flexible. The downside will be its resources footprint as it will scan all our mail to assign a score to them and basically >90% of them will be SPAM.

Others solution exists, more resources efficient, but with others downside tho. It’s the case of using RBL (Real-time Blackhole). It’s a database of known spammy IPs, from Spamhaus for example. You can select the spammy IPs list to block (Some are larger than others). However the downside is you might block legitimate IPs as only 1 domain might actually spam and all the rest on same IP could be legitimate. Or worse, in some cases, Spamhaus and co blocked a full range of IP…

But there is also others way to do it, like with Postscreen. As most of the Spam are sent by Zombies computers and have only a very limited amount of time to deliver their spammy mails before being backlisted, they tend to make compromises in their SMTP protocol implementation, for example, they may speak before their turn or they may ignore responses from SMTP servers and continue sending mail even when the server tells them no to do so, etc… In that case, Postscreen is here to see if they respect the SMTP protocol and if they do, will allow the mail to be delivered.

I think this process is quite efficient and could save a lot of resources as SpamAssassin will not have to scan all the mails, but only the one having passed the first tests from Postscreen. However if rejected, the client will need to resend the mail (Usually spammers don’t) and in this case you can have a long grace period (Several minutes to several hours depending on the client…). For this reason I do not use it but if you are having a heavy load due to spam and spamassassin don’t work enough or use all your resources, it’s a good turnaround.

http://upload.wikimedia.org/wikipedia/en/e/ed/Clam.pngAnother aspect to cover is having an AntiVirus. For linux? you will say? Well first of all, Linux is not perfect (Although it managed much better the authorization and system access than Windows) but you could suffer from some virus. But most important, you may not be the only user that will read mails coming from your server. Either you could offer access to family, friends, … or read your mails on different system including Windows or simply forward a mail to others people. That’s why I think having a proper AntiVirus for your mails is important.

But here again, having an AntiVirus that will scan all your mails to look for viruses will use a significant amount of resources  (30-50mb Ram probably?) and here is where again Postscreen could help, to avoid scanning Spam mails too.

Actually, to make this configuration works, you will also need an additional package, Amavis to buckle the loop:

Postscreen will remove at the earliest stage a significant part of Spam (The one not respecting SMTP protocol implementation) and let them go to Postfix. Amavis will then do the bridge between Postfix and SpamAssassin + ClamAV to check the Spam and Virus and finally Procmail to dispatch all these into the local mailbox. (Note that Sieve in Dovecot could do it too)

So let’s see how to install and configure all this.

PS: I don’t use Postscreen and if you want no delays in your mail, shouldn’t use.

Installation

apt-get install amavisd-new spamassassin clamav-daemon libnet-dns-perl libmail-spf-perl pyzor razor

and we will also add some compression tools to be able to scan the archives for viruses too.

apt-get install arj bzip2 cabextract cpio file gzip nomarch pax rar unrar unzip zip zoo

Postscreen is part of Postfix and does not require additional package.

Configuration

  • ClamAV:

Per default, ClamAV will automatically update its database every hour. If you want to update it now, you can run:

freshclam

Then, to avoid ownership issues during scans from ClamAV and Amavis, we need to add ClamAV and Amavis users to each others’ groups:

adduser clamav amavis
adduser amavis clamav
  • Amavis:

You will need to make Amavis and Postfix communicate.

In /etc/postfix/master.cf, below the line:

pickup    fifo  n       -       -       60      1       pickup

add:

         -o content_filter=
         -o receive_override_options=no_header_body_checks

to looks like that:

pickup    fifo  n       -       -       60      1       pickup
         -o content_filter=
         -o receive_override_options=no_header_body_checks

And at the end of the file add:

# Options for the filter
smtp-amavis     unix    -       -       -       -       2       smtp
        -o smtp_data_done_timeout=1200
        -o smtp_send_xforward_command=yes
        -o disable_dns_lookups=yes
        -o max_use=20

# Listener for filtered mail
127.0.0.1:10025 inet    n       -       -       -       -       smtpd
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_delay_reject=no
        -o smtpd_client_restrictions=permit_mynetworks,reject
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o smtpd_data_restrictions=reject_unauth_pipelining
        -o smtpd_end_of_data_restrictions=
        -o mynetworks=127.0.0.0/8
        -o smtpd_error_sleep_time=0
        -o smtpd_soft_error_limit=1001
        -o smtpd_hard_error_limit=1000
        -o smtpd_client_connection_count_limit=0
        -o smtpd_client_connection_rate_limit=0
        -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

then in /etc/postfix/main.cf, add:

content_filter = smtp-amavis:[127.0.0.1]:10024

Now you need to configure Amavis directly. In /etc/amavis/conf.d/15-content_filter_mode, make sure the 2 variables

@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);

 @bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

are uncommented. You’re now good to go to SpamAssassin

  • SpamAssassin:

I suggest to create a dedicated user to run spamassassin to better control the process and have dedicated logs.

In root (su) type:

groupadd spamd
useradd -g spamd -s /bin/false -d /var/log/spamassassin spamd
mkdir /var/log/spamassassin
chown spamd:spamd /var/log/spamassassin

Its configuration file is located in /etc/default/spamassassin. You will need to modify few things to enable SpamAssassin:

sudo nano /etc/default/spamassassin

and change the following to 1

ENABLED=1
CRON=1

You will also need to modify the OPTION line to become:

OPTIONS="--create-prefs --max-children 2 --username spamd -H ${SAHOME} -s ${SAHOME}spamd.log"

and add a new line with:

SAHOME="/var/log/spamassassin/"

Now you need to configure Postfix to use SpamAssassin

nano /etc/postfix/master.cf

At the line:

smtp      inet  n       -       -       -       -       smtpd

add below (new line):

-o content_filter=spamassassin

then at the end of the file, add:

spamassassin unix -     n       n       -       -       pipe
        user=spamd argv=/usr/bin/spamc -f -e  
        /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Finally restart all the services you have touched to.

/etc/init.d/amavis restart
/etc/init.d/spamassassin restart
/etc.init.d/postfix restart
/etc/init.d/dovecot restart

If any issue happen during the restart, it should tell you what to do. If no issue, you should now be protected from Spam and Viruses.

You can try if it works by sending a fake spam to your mail box. Simply send you an email with the content:

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

or try with a inoffensive virus from The European Expert Group For IT-Security.

  • Procmail:

You may want to make sure they are store in your Junk box to separate them from your regular inbox. Here is where Procmail enter. (Although Sieve in Dovecot could do the same)

First, you will need to tell postfix to use procmail.

nano /etc/postfix/main.cf

add the following line:

mailbox_command = /usr/bin/procmail -Y -a $DOMAIN

then, we need to config the rules.

From the Dovecot wiki, it states that Procmail seems to have some intermittent delivery problems if you use the system-wide configuration with Maildir style mailboxes. (/etc/procmailrc) and thus should use $HOME/.procmailrc instead.

Hence, to avoid having to configure that at every new email/user we will use the skel system to ensure our .procmailrc is copied to every new user.

In root, create the /etc/skel/.procmailrc file

nano /etc/skel/.procmailrc

and copy this simple configuration:

SHELL="/bin/bash"
MAILDIR="$HOME/Maildir/"
DEFAULT="$HOME/Maildir/"

JUNKMAIL=$HOME/Maildir/.Junk/

LOGFILE=$HOME/procmailrc.log
VERBOSE=yes
LOGABSTRACT=all
DROPPRIVS=yes

# Procmail rule to delete spam
:0:
* ^X-Spam-Status: Yes
$JUNKMAIL

This will route the SPAM in the .Junk folder. (You should be able to subscribe to this folder using your favourite email client like Thunderbird,…)

When you will create a new user, the user will have this .procmailrc in its home and should be able to have it email running directly.

As explained in the first part of this tutorial, to create a new user: (In root)

useradd --create-home -s /sbin/nologin youruser
passwd youruser

A long tutorial but you should now have access to a secure mail system.

A New CAPTCHA Approach

If you want to use Postscreen to have an additional layer of Spam protection, you can follow below tutorial:

  • Postscreen:

In your /etc/postfix/main.cf, add a section for Postscreen as following:

# Postcreen configuration

postscreen_greet_banner = Please wait to be seated
postscreen_greet_action = enforce

postscreen_pipelining_enable = yes
postscreen_pipelining_action = enforce

postscreen_non_smtp_command_enable = yes
postscreen_non_smtp_command_action = enforce

postscreen_bare_newline_enable = yes
postscreen_bare_newline_action = enforce

Few explanation:

greet_banner

When a client connect to Postscreen, it will start to communicate by sending a first banner “Please wait to be seated” and 6 seconds later, the remaining information on the SMTP identity. According to SMTP protocol, the client needs to wait to receive the entire banner. Spam bots will probably not wait (as they are configured to send as many mails as possible) and Postscreen will not accept its mail.

pipelining_enable

Initially, before the ESMTP (Extended SMTP), the protocol was half-duplex, mining the server and client needed to send 1 command at a time and wait for the answer of the other. Enabling this option will indicate to the client that he needs to send 1 command at the time as Postscreen “does not” support ESMTP. Here again, most probably Spam bots will not respect that and send the entire set of commands directly.

non_smtp_command_enable

This test is a simple filter that block the commands CONNECT, GET and POST, used by spam bots when they use proxies. This filter is actually already implemented in Postfix (Since version 2.2) but having at the upstream should help reduce the load on the smtp daemon.

bare_newline_enable

This test is still very simple but a lot of Spam bots don’t respect it….in the SMTP protocol implementation, each line should finish by for “Carriage Return & Line Feed”. But a lot of zombies only use the at the end of their line.

Obviously many more options exists and you should read the official documentation to learn more.

Then you need to modify the /etc/postfix/master.cf to enable Postscreen and allow him to route the validated mails to smtpd.(In root)

nano /etc/postfix/master.cf

and replace the line

smtp      inet  n       -       -       -       -       smtpd

by

smtp      inet  n       -       -       -       1       postscreen
smtpd     pass  -       -       -       -       -       smtpd
dnsblog   unix  -       -       -       -       0       dnsblog

and then restart postfix

/etc/init.d/postfix restart

However you will receive mails with a delay from few minutes (5mn from Hotmail and 20mn from Gmail based on my previous test) to few hours depending on the client side….that’s why I don’t use Postscreen in fact.

Gravatar

Loves to discover web-based apps to install on his own server@home and write articles about it

0 Comments:

Add a comment