Daniel J. Dick

I just  spent a week in email purgatory.

Sendmail was a great email server.  It served millions of people for decades.  But, email technology has matured.  Or perhaps it has merely been simplified.  Or perhaps complexity for the sake of generality has been exchanged for a simple foundation with complex appendages to address security, performance, and scalability.

Within minutes of receiving my virtual private server (VPS) account, I installed Apache, PHP, an APC cache, WordPress, Drupal, and a few security plugins for each and immediately noticed about 60,000 spambot attacks.  And it did not even have a hostname in DNS attached to it.

I grabbed an sshblacklist Perl script and began to drop these bad guys into the firewall.

So, when I wanted to move my email across from my old C-panel hosting, I trembled and feared.  Sendmail was what I used in the past, and it was old and not considered very secure.  I considered qmail which was great for performance and security, but it was not open source and it was unmaintained.  Postfix almost seemed to be in the same place, and  both qmail and postfix were considered the most secure email servers around.  But, then there was Exim which was almost as secure and theoretically could be configured properly to be as secure.  It had a huge supportive user community and was very flexible.  So I installed it.

One excellent, informative article on the differences between these four major email systems can be found at  http://shearer.org/MTA_Comparison.

Configuring Exim

Configuring Exim looked easier.  The rules were more human readable than sendmail’s configuration files though I had become accustomed to regular expressions used by sendmail, Perl, lex, yacc, and so many other environments.

So, I took a plain-Jane, out of the box configuration, made a few tiny tweaks and had a server up and running with my first accounts on it.  Accounts.  Hmmm.  For which domain?  Godsblogs.com?  Nodivorces.com?  DanielJDick.com?  Furthermore, it just did not seem safe to have your email accounts match your system’s accounts.  Hackers would assume if email flows in that there must be a unix account behind it.  Why not authenticate in MySQL or LDAP or something that can handle multiple domains?  I chose MySQL.

MySql

I set up a couple database tables in MySQL–one for users and one for domains.  Then I coded them into the exim.conf file.

You can find examples all over the web including one at http://maxo.captainnet.net/installs/tiger/ECM/exim_4.60.configure.txt

It became trivial to add new users and aliases and domains to the environment and to set up TLS/SSL with authentication so I could log in from my laptop using Mozilla’s Thunderbird or Outlook Express.

Moving Time – Mailbox and Maildir

After pointing DNS to my VPS server and having my emails flowing in, I needed to move my old emails across.  But, Thunderbird seemed slow when accessing email on my new server, and that didn’t make sense at all.

Then I realized I did something dummmmb.  All my new emails were in Mailbox files while the email on my old C-panel service was in Maildir files–one file per email instead of all emails crammed sequentially into one file.   So, I renamed the old directories for each domain and created new mail directories based on Maildir and reconfigured Exim.  And things sped up considerably.

Merging Email

Now I had tens of thousands of emails on the old system in Maildir format, a few hundred perhaps in Mailbox format, and the new emails in Maildir format on the new system.  So, how to get them all into one area?

Well, I found a Perl or Python script called mb2md and I had to rework it considerably.  It was anchored to the old notion that email accounts always sit directly in /var/mail and that their ownership is always paired up with the system account that would always be found in /etc/passwd.

I fixed the script for my system configuration and ran them one by one to get all the mailbox files into their appropriate Maildir directories, and it worked great except for one thing:  the files were owned by root.  I fixed that.  Then I went to thunderbird and moved some of the subdirectories under INBOX, and they disappeared.  I had gotten tired of griping and complaining and getting angry and I was beginning to make my peace with the loss of those files until I realized they were in dotted or hidden directories.  Doh! strikes again.

In short, I got all the email moved, and I was a very happy camper.

Monkeys on the System

Spam came in.  That was expected.  I turned off spamassassin to save memory.  It was either that or allow memory to be swallowed until one of the webservers would crash.  Funny thing is I set up scripts to check and restart them, but there had to be a better solution.

I Googled again for good, fast ways to eliminate spam.  Some of them I had already coded into my Exim installation.  Relaying was blocked.  Certain characteristics of the headers or body were strong indicators of spam, so those were eliminated.

But, there was a bigger problem – relaying attempts

Spam idiots love to steal resources on other machines to do their bidding.  Two ways they do it are:

1. Install a spambot onto your system through a trojan horse or some kind of virus.
2. Send spam through your web server to other people using it as a “relay”.

In this case, a few hundred spambots on other people’s machines were trying to use my email server for relaying spam.  That is, they would attempt to send millions of spams through my server to other people.

But, my system did not allow relaying.

Even so, when I went to the error logs, I found that my main.log and reject.log file were growing to dozens of megabytes in no time.  That was eating up my networking resources, burdening my email server, and swallowing up diskspace by filling my logs full of error messages.

So, I grabbed a copy of a Perl program I used earlier called sshblacklist.pl.  I ripped out the guts of the program so it would drop the offenders into the firewall permanently and quickly and then check for duplicate entries and eliminate them from the firewall.  That kept the firewall down to about 600 entries rather than 2400 or so.  And the log files stopped growing so rapidly.

Life seemed peaceful once again.

But, then there was spam

Before this time, I slapped SpamAssassin onto the system.  It was a very popular, intelligent, and effective spam detector.  But, it required memory–more memory than my little VPS could afford.

So, I Googled for alternatives.  And I got some irrelevant results, so I Binged for alternatives and Yahooed for alternatives and found myself stuck wondering which one I liked best.

The best, most effective, most popular, and most resource economical solution I found was “greylisting”.  Another very easy and effective solution was to include a little block of code into the Exim configuration to check all IP addresses with spamhaus and check the domain names with a few other lists to see if they were known spammers.  I turned that on and felt terribly guilty that my email server was hammering spamhaus and the other sites so often for the same sites over and over again, so I turned that off until I could lower the traffic of requests internally.  Another solution I used with success long ago was to use something called Boxtrapper.

Boxtrapper

What is boxtrapper?  Well, it’s cool.  Very cool.  And for a period of time,. it drove my spam down to zero while allowing all legitimate email traffic through.

How did it work?

Boxtrapper kept a few lists:  a blacklist, a whitelist, and another form of blacklist.  If I remember right, the two blacklists would be for sending denial messages for one and just cutting off the other with no response.  And the whitelist, of course, would be there for emails that should be allowed to flow through unhindered.

So, what was different about Boxtrapper?  If you sent an email to someone who was protected by Boxtrapper, if Boxtrapper did not know you, it would send an email back to you asking you to reply just to verify that you were really a human being sending a real email.  Spambots would not know how to reply, but people would.  So, once you replied, you would be listed in the whitelist and you would never have to send another reply to prove you were human again.

Unfortunately, Boxtrapper came with C-panel and did not live outside C-panel, and many administrators experienced problems managing servers with Boxtrapper installed, so they began to stop boxtrapper services telling people to use SpamAssassin instead.

So, Boxtrapper was not an option for me.

How About Greylisting?

What is “greylisting”?  Like Boxtrapper, greylisting may have a whitelist and a blacklist.  But, they have another list in-between.  Anyone care to guess what it might be called?

Greylisting is like Boxtrapper without human intervention.  When you send email to someone who is protected by greylisting, the other system won’t know who you are.  So, it tells your email system you are not authorized to send email to that person.  Yet.  So your system needs to come back and try again sometime later.  Perhaps it’s five minutes or fifteen minutes or an hour.  Real email servers will almost always do that, and spamming systems almost never retry.  But, some do.  So you get rid of most of the spam.

I noticed two popular and effective solutions:

• The greylistd system
• Coding greylisting directly into Exim’s configuration file.

I tried both a little but I liked the greylistd solution best.  With greylistd, I found it almost trivial to configure it into Exim and it provided programs for analyzing and manipulating the blacklist, whitelist, and greylist.

Summary

During the process, I was terrorized by several agonizing daggers to my heart as I despaired over how to convert various files into different forms, feeling torn between losing email and getting spammed, and suddenly finding all my past email hiding in a hidden directory apparently gone forever.

But, it turned out to be a great learning experience, and in the end I had an awesome email set up.