A multi-domain mail server on Debian

09 okt 2005

This is a description of how to set up a Debian box to serve as a multi-domain mail server, with SMTP (authenication available), POP3, webmail and mailing lists.

It is basically a modified and extended version of the description found here:

DovecotPostgresql

[Note: This description is no longer being maintained so it may be somewhat depracated. Also, rather than setting up a service like this from scratch you might want to take a look at assisted solutions like ISPConfig.]

This solution has two features that are essential if you want to serve more than just a few users: a) email accounts that are not related to user accounts on the server (unless you want them to) and b) support for multiple domains. It also keeps all of the addresses, aliases etc in a database which may be a good thing if you need to change those relatively often and you think you may want to build an admin interface for your email server. Another great feature is that you can create new mailing lists without manually adding those addresses.

We’ll use the following programs for the task: Postfix for SMTP, Dovecot for POP3, SASL2 for SMTP authentication, SQWebmail, PostgreSQL for user/address info and Mailman for mail lists. All running on Debian—in my case I use Etch (currently “testing”) but I’ve had it running on Sarge (now “stable”) as well. Of course, the same software can run on other flavours of Linux as well but there would probably be some differences.

First, the mandatory disclaimer: Use this instructions at your own risk. I won’t take responsibility if something goes wrong. (And it won’t hurt if you do some extra research on your own if you’re unsure about something. Please report any errors you might find.)

You need some experience with Linux and SQL to be able to follow the instructions. It will probably take some time and effort before you’re done. Remember, web hosting is cheap these days and running your own mail server may not be the smartest thing to do. But if you’re willing to experiment, let’s go!

Install packages

First, we need to install the required packages. Below is a list of packages which I compiled while going through the different parts of the installation. These packages will depend on other packages that will be installed at the same time.

postfix-tls if you’re using Debian Sarge/stable or:
postfix for Debian Etch/testing
postfix-pgsql
sasl2-bin
libsasl2-modules
postgresql
libpam-pgsql
dovecot-pop3d (you might also want to use dovecot-imapd although that is not covered here)
courier (which is a base package for sqwebmail)
sqwebmail
courier-authpostgresql if you’re using Debian Sarge/stable or:
courier-authlib-postgresql for Debian Etch/testing
mailman

Please note the difference between postfix and postfix-tls above. In Sarge there were two versions of Postfix: one with support for smtp-auth and one without. In Etch the tho have been merged into one package. Also, the package courier-authpostgresql in Sarge is replaced by courier-authlib-postgresql in Etch.

(Note: right now there is a bug in the dovecot packages which can be temporarily solved by installning the package zlib1g-dev.)

Install these packages however you prefer on your Debian box, for example: apt-get install postfix-tls postfix-pgsql ... (executed as root).

User accounts for mail handling

I said earlier that we won’t be required to keep user accounts for each email account. However, it may be a good idea to have one user account for each domain that you host. That way, the messages belonging to people in one domain will be unreachable by all other users under all circumstances. Example (again, run as root):

adduser domain1
adduser domain2

You may also let your users use these accounts to manage their web space etc.

Setting up Postgresql

You need to tell Postgresql to allow local connections using password authentication. Change the appropriate line in /etc/postgresql/pg_hba.conf to:

host    all    all    127.0.0.1    255.255.255.255    password

Now, let’s create the database. First we’ll su to the postgres user to get the necessary permissions. (You may have other means of doing this.)

sudo su postgres
createdb mail_info

Connect to your database with the psql command line program (or with your preferred method, phppgadmin for instance):

psql mail_info

OK, it’s time for some SQL business. First run this SQL code to create the tables (and two views):

CREATE TABLE transport (
domain VARCHAR(128) NOT NULL,
transport VARCHAR(128) NOT NULL,
PRIMARY KEY (domain)
);

CREATE TABLE mailuser (
userid VARCHAR(128) NOT NULL,
password VARCHAR(128),
realname VARCHAR(128),
uid INTEGER NOT NULL,
gid INTEGER NOT NULL,
home VARCHAR(128),
mail VARCHAR(255),
PRIMARY KEY (userid)
);

CREATE TABLE virtual (
address VARCHAR(255) NOT NULL,
userid VARCHAR(255) NOT NULL,
PRIMARY KEY (address,userid)
);

CREATE VIEW postfix_mailboxes as
SELECT userid, home||'/' as mailbox from mailuser
UNION
SELECT domain, 'dummy' from transport;

CREATE view postfix_virtual as
SELECT userid, address from virtual
UNION
SELECT userid, userid as address from mailuser;

A suggestion at this point is to create database user that has permissions to read these tables and nothing else. You can also create a user that has permissions to edit the data if you don’t want to use the postgres user (or some other priviliged user) for that.

CREATE USER mailreader PASSWORD 'xxxxxxx';
GRANT select ON
transport, mailuser, virtual, postfix_mailboxes, postfix_virtual
TO mailreader;

CREATE USER mailadmin PASSWORD 'xxxxxx';
GRANT select, insert, update, delete ON
transport, mailuser, virtual, postfix_mailboxes, postfix_virtual
TO mailadmin;

Next, time to insert some basic values to get you started. We’ll add records for the imaginary domain1.com. Edit as appropriate.

INSERT INTO transport (domain, transport)
VALUES ('domain1.com', 'virtual:');
INSERT INTO mailuser (userid, password, uid, gid, home)
VALUES ('webmaster@domain1.com', 'secret', 1002, 1002, 'domain1.com/webmaster');

The first record tells Postfix to handle this domain with the Virtual method. The second adds a mail account. Please note:

The values ‘1002’ and ‘1002’ are the uid and gid that will be used to access your actual mail data. Set this according to the users that you created for your different domains. (Remember?)
The value ‘domain1.com/webmaster’ is the relative path to the directory where the mail data will be stored. You’ll need to follow this pattern exactly. We’ll choose where to store the mail data later.
The password (‘secret’) are stored as plain text. There are other methods, such as md5, but I never got them to work with all of the components of the mail system.

Let’s move on:

INSERT INTO virtual (address, userid)
VALUES ('hostmaster@domain1.com', 'webmaster@domain1.com');
INSERT INTO virtual (address, userid)
VALUES ('domainmaster@domain1.com', 'webmaster@domain1.com');
INSERT INTO virtual (address, userid)
VALUES ('sysadmin@domain1.com', 'webmaster@domain1.com');
INSERT INTO virtual (address, userid)
VALUES ('postmaster@domain1.com', 'webmaster@domain1.com');
INSERT INTO virtual (address, userid)
VALUES ('abuse@domain1.com', 'webmaster@domain1.com');
INSERT INTO virtual (address, userid)
VALUES ('webmaster@domain1.com', 'forward.to@myotheraddress.com');

The “virtual” table lets you do two things: create aliases that forward mail to specific addresses (record 1-5) and forward copies of mail from your accounts to other addresses (record 6). Here I have added some addresses that people may expect to be able to contact you by.

Repeat the steps above for each domain that you want to serve.

Next, we need to define how external mail (mail to domains other than those that you host) should be handled. I’m using an external smtp server, but that might not be what you want. Check the Postfix documentation for more info.

insert into transport (domain, transport)
VALUES ('*', 'smtp:[smtp.myisp.com]');

Local mail (such as cronjob reports) is delivered directly to the appropriate user. Add your local hostnames:

insert into transport (domain, transport)
VALUES ('localhost', 'local:');
insert into transport (domain, transport)
VALUES ('myhostname', 'local:');
insert into transport (domain, transport)
VALUES ('mail.mydomain.com', 'local:');

We’re done! Exit the psql program by entering ’\q’.

Configure Postfix

This is my version of the config file /etc/postfix/main.cf:

#See /usr/share/postfix/main.cf.dist for a commented, more complete version

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

#appending .domain is the MUA's job.
append_dot_mydomain = no

#Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

myhostname = mail.yourdomain.com
mydomain = yourdomain.com
myorigin = /etc/mailname
mydestination = mail.yourdomain.com, localhost, myhostname

alias_maps = hash:/etc/aliases
#alias_database = hash:/etc/aliases
mynetworks_style = subnet # works with Etch only?
#mynetworks = 127.0.0.0/8 XXX.XX.XX.0/8 # may be used instead
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
recipient_delimiter = +
delay_warning_time = 2

#Mailman mailing lists
#http://www.gurulabs.com/downloads/postfix-to-mailman-2.1.py
relay_domains = lists.yourdomain.com
mailman_destination_recipient_limit = 1

#Virtual mailbox config
transport_maps = pgsql:/etc/postfix/transport.cf, hash:/etc/postfix/transport_mm
virtual_uid_maps = pgsql:/etc/postfix/uids.cf
virtual_gid_maps = pgsql:/etc/postfix/gids.cf
virtual_mailbox_base = /var/mail/vhosts
virtual_mailbox_maps = pgsql:/etc/postfix/mailboxes.cf
virtual_maps = pgsql:/etc/postfix/virtual.cf
mydestination = $mydomain, $myhostname

#SASL auth config
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = yourdomain.com
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination

#In order to relay to a smtp-auth server:
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options =

You need to look through this file carefully and change the parts that are specific for your setup. The Postfix documentation may come in handy.

A few notes:

virtual_mailbox_base = /var/mail/vhosts: this is where your mail data will be stored. You need to create the vhosts directory (or whatever you choose). Let it be owned by root. In that directory you’ll have one directory per domain, named after the domains, so go ahead and create those. Change ownership to each domain’s user. Then, you need one directory per user, named by the user part of the addresses (user@domain.com). These too should be owned by the domain users.
SASL auth is configured so that you are required to log in to the SMTP server in order to send email to external domains, unless you are on the same subnet as the server (mynetworks/mynetworks_style).
The domain lists.yourdomain.com is defined as the handler of all Mailman stuff.

Now, in /etc/postfix/master.cf, add this line to enable the Mailman domain:

mailman unix    -       n       n       -       -       pipe
        flags=FR user=list:list
        argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${user}

... and in /etc/postfix/sasl_passwd – put the password for the external SMTP server where you want to relay your external mail.

smtp.myisp.com        username:password

We need to build a .db file from sasl_passwd. Run this in a shell:

postmap /etc/postfix/sasl_passwd

Edit /etc/postfix/sasl/smtpd.conf:

pwcheck_method: saslauthd
saslauthd_path: /etc/mux

OK, next we’ll tell Postfix how to find the information it needs from our mail database. Note that this part also differs depending on your Debian version. If you’re using Etch you probably have a Postfix version above 2.1.

First, /etc/postfix/transport.cf:

user=mail_db_user
password=secret
dbname=mail_info
hosts=localhost

##For postfix > 2.1
query = SELECT transport FROM transport WHERE domain='%s'

##For postfix < 2.2
#table=transport
#select_field=transport
#where_field=domain

And in /etc/postfix/uids.cf:

user= mail_db_user
password=secret
dbname=mail_info
hosts=localhost

##For postfix > 2.1
query = SELECT uid FROM mailuser WHERE userid='%s'

##For postfix < 2.2
#table=mailuser
#select_field=uid
#where_field=userid

...and /etc/postfix/gids.cf:

user=mail_db_user
password=secret
dbname=mail_info
hosts=localhost

##For postfix > 2.1
query = SELECT gid FROM mailuser WHERE userid='%s'

##For postfix < 2.2
#table=mailuser
#select_field=gid
#where_field=userid

/etc/postfix/mailboxes.cf:

user=mail_db_user
password=secret
dbname=mail_info
hosts=localhost

##For postfix > 2.1
query = SELECT mailbox FROM postfix_mailboxes WHERE userid='%s'

##For postfix < 2.2
#table=postfix_mailboxes
#select_field=mailbox
#where_field=userid

/etc/postfix/virtual.cf:

user=mail_db_user
password=secret
dbname=mail_info
hosts=localhost

##For postfix > 2.1
query = SELECT userid FROM postfix_virtual WHERE address='%s'

##For postfix < 2.2
#table=postfix_virtual
#select_field=userid
#where_field=address

That’s it. Now you might want to add in /etc/aliases:

domain1: webmaster@domain1.com
domain2: webmaster@domain2.com

...which means that if there is local mail to the user accounts of your different mail domains, it will be forwarded to the appropriate address. After changing /etc/aliases you have to rebuild aliases.db by running:

newaliases

Finally, define the transport mechanism for mailman, in /etc/postfix/transport_mm:

lists.yourdomain.com   mailman:

...and build a corresponding .db file:

postmap /etc/postfix/transport_mm

SASL configuration

/etc/default/saslauthd:

START=yes
MECHANISMS=pam
PARAMS="-r -m /var/spool/postfix/etc"

Create /etc/pam_pgsql.conf as:

database = mail_info
host = localhost
user = mail_db_user
password = secret
table = mailuser
user_column = userid
pwd_column = password
#expired_column = acc_expired
#newtok_column = acc_new_pwreq
pw_type = plain
#debug

Create /etc/pam.d/smtp as:

auth        required    pam_pgsql.so 
account     required    pam_pgsql.so
password    required    pam_pgsql.so

Add to /etc/postfix/sasl/smtpd.conf:

mech_list: login plain

Configure Dovecot pop3 server:

In /etc/dovecot/dovecot.conf change to this:

protocols = pop3
[...]
default_mail_env = maildir:/var/mail/vhosts/%d/%n

Then, if you’re using Debian testing/Etch:

disable_plaintext_auth = no
[...]
pop3_uidl_format = %08Xu%08Xv
[...]
passdb sql {
args = /etc/dovecot/dovecot-sql.conf
}
[...]
userdb sql {
args = /etc/dovecot/dovecot-sql.conf
}

(Not sure why pop3_uidl_format has to be set, but after a recent version upgrade POP stopped working until I changed this.)

And this is the corresponding for Debian stable/Sarge:

auth_userdb = pgsql /etc/dovecot-pgsql.conf
auth_passdb = pgsql /etc/dovecot-pgsql.conf

Next, we edit the database config file. For testing/Etch, use /etc/dovecot/dovecot-sql.conf and for stable/Sarge use /etc/dovecot/dovecot-pgsql.conf:

driver = pgsql # Only for testing/Etch!
connect = host=localhost dbname=mail_info user=mail_db_user password=secret
default_pass_scheme = PLAIN
password_query = SELECT password FROM mailuser WHERE userid = '%u'
user_query = SELECT '/var/mail/vhosts/'||home, uid, gid FROM mailuser WHERE userid = '%u'

Setup SQwebmail

Change in /etc/courier/authdaemonrc:

authmodulelist=&#8221;authpgsql&#8221;

Change in /etc/courier/authpgsqlrc:

PGSQL_HOST              127.0.0.1
PGSQL_PORT              5432
PGSQL_USERNAME          mail_db_user
PGSQL_PASSWORD          secret

PGSQL_DATABASE          mail_info
PGSQL_USER_TABLE        mailuser

#PGSQL_CRYPT_PWFIELD   crypt
PGSQL_CLEAR_PWFIELD     password

PGSQL_UID_FIELD         uid
PGSQL_GID_FIELD         gid

PGSQL_LOGIN_FIELD       userid

PGSQL_HOME_FIELD        '/var/mail/vhosts/'||home
PGSQL_MAILDIR_FIELD     '/var/mail/vhosts/'||home

Restart processes

In a shell (as root):

/etc/init.d/saslauthd restart
/etc/init.d/postgresql restart
/etc/init.d/postfix restart
/etc/init.d/dovecot restart
/etc/init.d/courier-authdaemon restart

When that is done, everything should be ready! Now, try it out:

Setup a new account in your mail application, for example ‘webmaster@domain1.com’. The username is the same as the complete address. Try to send mail to and from that account.
Go to http://www.yourdomain.com/cgi-bin/sqwebmail and try out the web mail. Log in with the complete email address as user name.
Setup a mailing list with Mailman using an address under lists.yourdomain.com and test it.

Get ready for prime time

Make sure you’ve tested everything properly before releasing the service to your users. Remember to:

Check the DNS and make sure you have proper MX entries for your domains.
Check that all the related ip ports are open on your server. SMTP uses 25 and POP3 uses 110. And don’t forget the web interfaces of SQwebmail and Mailman—open port 80.
It’s a good thing to register an abuse@domain email address for each domain at abuse.net. Also make sure that those addresses resolve so that you receive any messages that may be sent to them.
Send messages to and from internal and external addresses, as well as between internal addresses, to make sure everything works ok.

Troubleshooting

Check /var/log/mail.log for errors. It usually gives you an idea about what went wrong.
Check file permissions of the mail data.

Requested features

Some things that would be nice to add:

An interface for administrating domains, accounts and aliases.
Spam filtering.
Virus scanning.

Revision history

29 Sept 2006: Recent versions of Postfix in Etch require some changes to some of the config files. The syntax for specifying the database tables and columns has changed.
7 Jul 2006: The package courier-authpgsql has been replaced by courier-authlib-postgresql in Etch.
4 Jan 2006: After upgrading Dovecot it suddenly broke. Log messages said I had to set pop3_uidl_format in dovecot.conf, which I did. So I added this to the guide.
1 Jan 2006: As Peter Nilsson points out, you have to CREATE a table before you GRANT access to it. Made some other small changes as well.