Tag: qmail

(auto magically) Deleting old Spam emails

If you followed My Qmail installation guide, or are using some kind of webmail or IMAP client that puts “SPAM” marked emails in some special folder, you can provide auto delete of old spam emails.

The popular Roundcube webmail has the movespam plugin (actually it’s broken, but is an easy fix) that moves spam to …/user/Maildir/.Junk folder. And this makes very easy to have a cron invoked script that deletes old spam emails.

So (in PHP), and with a threshold of 30 days (older are deleted):

$threshold = time() - (60*60*24*30); // 30 days

$junk_folders = shell_exec('find /home/vpopmail/domains/ -name .Junk -type d');
$junk_folders = explode("\n", $junk_folders);
$junk_folders = array_filter($junk_folders);

foreach ($junk_folders as $junk_folder) {
  foreach (array('new','cur') as $subfolder) {
    $d = dir($junk_folder.'/'.$subfolder);
    while (false !== ($entry = $d->read())) {
      if ($entry != '.' && $entry != '..' && is_file($d->path.'/'.$entry)) {
        if (filemtime($d->path.'/'.$entry) < $threshold)
          unlink($d->path.'/'.$entry);
      }
    }
  }
}

When we are deleting emails directly like this, you are screwing the user quota so probably it’s not a bad idea to rebuild quotas after running the old spam delete script.

Nasty qmail-remote hangs forever bug

I am very happy with my current mail setup. But a nasty bug popped up out of nowhere and i can’t trace it…. Some qmail-remote processes, in some circunstances yet to determine, just hang-up forever eating all the available CPU. The qmail-remote is the piece of Qmail that takes care of message delivery to recipients at a remote host.

When this happens, the stuck process doesn’t conform to the timeoutremote control and stays active forever. The truss command (FreeBSD equivalent to strace) doesn’t show any activity and neither appears to be related network activity… it looks like some kind of race condition.

For now i couldn’t really address the issue, to both lack of time and deep understanding about C and debugging with GDB, so i just mitigate the problem with a cron running a bash script to detect and kill the offending processes.

#!/usr/local/bin/bash

# limit in seconds
LIMIT_TIME=120
# limit in cpu
LIMIT_CPU=35


IFS=$'\n'

for line in `ps -xao pid,etime,command,%cpu | grep qmail-remote`; do

  pid=`echo $line | awk '{split($0,a," "); print a[1]}'`
  time=`echo $line | awk '{split($0,a," "); print a[2]}'`
  cpu=`echo $line | awk '{split($0,a," "); print a[5]}'`
  cpu=`echo "($cpu)/1+1" | bc`

  IFS=$':'
  time_parts=($time)

  if [ ${#time_parts[@]} -lt 3 ]; then
    elapsed=`echo ${time_parts[0]}*60 + ${time_parts[1]} | bc`
  else
    elapsed=`echo ${time_parts[0]}*3600 + ${time_parts[1]}*60 + ${time_parts[2]} | bc`
  fi

  if [ $elapsed -gt $LIMIT_TIME -a $cpu -gt $LIMIT_CPU ]; then
    kill -s 9 $pid
   fi

IFS=$'\n'
done

But i’m not really happy with this “solution”, and will be pursuing a real understanding and solution for this proble.

Some interesting links about other people with the same problem:

http://permalink.gmane.org/gmane.os.freebsd.stable/82760
http://copilotco.com/mail-archives/qmail.2002/msg08733.html

UPDATE and SOLUTION

All credits, to where credits are due.
To replicate this, you should catch an hanging qmail-remote with top. Then filter the offending qmail-remote pid trough ps to get full arguments list:

ps -wwaux | grep pid_number

You should get something like ‘qmail-remote mailserver from@email to@email’. With this information, and with top and truss you can invoke qmail-remote from the command line and get a nice qmail-remote hang…

truss /var/qmail/bin/qmail-remote mailserver my@email nonexisten@email < ./test

test is just a file with some bogus input to serve as the sending email, as qmail-remote expects a message in stdin. Just for the note, most of the hangs happens when talking to the Symantec Email Gateway software.

Now, with an updated ports tree, just recompile qmail, you can follow my guide (all good there). But, just issue make, no need to make install. Then move qmail-remote to /var/qmail/bin/ and set the right permissions (711) and ownership (root:qmail).

And voilá, if you repeat the test procedure, you will find that qmail-remote is not hanging anymore 🙂

My Qmail installation guide

A fresh Qmail installation on FreeBSD is something that i have to deal once a couple of years. It´s just one of those things that i could well live without, but the time will come again and usually i can’t remember of half of my previous installation…. this is my personal installation guide to do it faster and with less effort.

Why Qmail? Why FreeBSD? Well, if you come all the way into this dark corner of the Internet, you should know the answer to both… so moving on, this is a massive, uber-geek, fully comprehensive and detailed installation guide, so it takes time and some brain damage.

CAUTION: proceed at your own risk

Continue reading “My Qmail installation guide”

Pages: Page 1, Page 2

Qmail + Vpopmail – Rebuild maildirsize file

Let’s say some program or script that doesn’t support maildirquota was poking around with a user maildir and you need to recalculate the user maildirsize file (recalculate the quota), you just need to:

cd /home/vpopmail/domains/domain.com/user/Maildir
rm maildirsize
vuserinfo -Q user@domain.com

and, voilá a new and accurate maildirsize file!

EDIT

I did a very simple script that rebuilds the quotas system wide (yes.. it’s PHP, not in real man C or some other fashion language, but it works for me)

$vpopmail_bin = '/home/vpopmail/bin/';

$domains = dir('/home/vpopmail/domains');
while (false !== ($domain = $domains->read())) {
  if ($domain != '.' && $domain != '..' && is_dir($domains->path.'/'.$domain)) {
    $users = dir($domains->path.'/'.$domain);
    while (false !== ($user = $users->read())) {
      if ($user != '.' && $user != '..' && is_dir($users->path.'/'.$user)) {
        if (file_exists($users->path.'/'.$user.'/Maildir/maildirsize')) {
          unlink($users->path.'/'.$user.'/Maildir/maildirsize');
          exec($vpopmail_bin.'vuserinfo -Q '.$user.'@'.$domain);
        }
      }
    }
  }
}