Email Postfix in Practice

Categories: Linux

(Back to the main article)

(Updated 2024-04-14 to Postfix version 3.6.4 on Ubuntu 22.04 LTS)

Introduction

This article describes the steps needed to install Postfix + Dovecot + SpamAssassin on an Ubuntu server. The implemented solution is very light-weight, and suitable for handling mail for a small number of users (eg a single person, a family or small startup). In particular, user-accounts are defined via config-files (not a database), in order to use the least amount of resources.

The reasons for choosing such a setup are described here. A general overview of email is described here. Some background information on Postfix can be found here.

The instructions describe how to set up email for user@example.com on a server with name mail.example.com. Replace example.com with your domain-name.

The solution described here uses the Postfix virtual delivery transport, with email-accounts defined in a hand-editable plain text file.

Set up a Test Email Domain

The steps below are rather complicated, and getting it right first time is not easy. If you are migrating an existing email-domain from a hosted service to a self-configured system then it can be awkward to switch over and find the new system isn’t working right.

If you are migrating an existing email domain, then I recommend first setting up a new email system for a “test domain”, and switching the real domain over after confirming that emails via the test-domain work. Assuming you have a domain-name example.com, it is possible to apply the following instructions to set up email-domain @test.example.com, send and receive emails, then update to handle the real @example.com addresses - mostly just a case of replacing @example.com with @test.example.com below…

Configure a Host Name

As well as registering a DNS record for your email-domain, you need to register a DNS name for the host on which the SMTP server for that domain runs. I recommend creating an alias-record for mail.example.com which points at whichever host you want to deploy your SMTP server on. That is not only clean, but also allows you to restructure later with reduced impact - in particular the SSL certificate for the alias host name is still valid even if you move the underlying infrastructure to a new host.

See the section below on DNS configuration for more details.

Create an SSL Certificate

It is assumed that you have already obtained an SSL certificate for “mail.example.com” (replace example.com with your domain). Using Letsencrypt is a good option.

Configure the System Firewall

Ubuntu automatically uses UFW to block most incoming ports by default. Run ufw status verbose to see the current rules; if ufw is active then:

  • ufw allow 465 # smtp-over-tls
  • ufw allow 587 # submission
  • ufw allow 993 # imap-over-tls
  • ufw allow 220 # imap (hopefully with STARTTLS)

Install Dovecot

Before setting up Postfix to handle incoming email, we should set up the system that actually stores those emails on disk, and provides access to them. Postfix can store files on disk, but doesn’t support POP or IMAP protocols for retrieving them remotely - for that you need something like Dovecot. And if you’re using Dovecot then Postfix should not store incoming mails directly, but instead pass them to the Dovecot server as soon as they are received and validated.

Dovecot accepts emails from a Postfix instance (via lmtp) and stores them. It also handles email-client-applications connecting via the IMAP protocol to view/manage/delete existing emails.

Outgoing emails from an email-client-application point at the Postfix “submission port”, not at Dovecot.

Install Dovecot with:

  • apt install dovecot-core dovecot-imapd dovecot-lmtpd

In /etc/dovecot/conf.d, configure files as follows. Note that commented-out lines in the installed config-files indicate the default values.

Dovecot Auth

When using Postfix and Dovecot together, doing user authentication is a little tricky; Dovecot needs to authenticate users accessing email via IMAP/POP3, but Postfix also needs to authenticate users who send email via the “submission port” and needs to at least verify that a user exists when receiving email via SMTP from other systems.

There are a few ways to set up this “shared authentication”; both Dovecot and Postgres are very configurable in this area.

Larger systems (and most tutorials on email setup) use a relational database (eg MySQL or Postgres) to store users, and share this DB between Postfix and Dovecot. However this is obviously resource-intensive - the DB needs to run continuously.

It is possible for Postfix and Dovecot users to simply be local Linux users, ie to use standard user accounts on the mail host - and in fact the default “auth-system” settings for Dovecot assume this approach (via the PAM login system). However (a) that makes obtaining user-related info needed in other contexts difficult and (b) it seems a bad idea to link email passwords to login passwords.

This article recommends having a plain-text file for Dovecot user information and credentials. This is easy to understand and debug, and seems safer. Postfix can then authenticate users sending outbound mail via a “SASL” connector to Dovecot - ie ask Dovecot to do the user validation. Unfortunately, when handling incoming mail Postfix needs to check if the destination email account exists but cannot use SASL for that as there is no password available; the only solution I am aware of is a separate file for Postfix - ie duplicated info that needs to be kept in-sync.

Using simple file-based user authentication does mean that adding new users (or even changing their passwords) requires editing a text file on the mailserver; it is therefore appropriate for systems with a small number of users and a technically-minded admin. But if you are reading this article, that’s presumably you..

Edit /etc/dovecot/conf.d/10-auth.conf to:

  • include only auth-passwdfile.conf.ext (see end of file)
  • set auth_username_format = %Ln (alternative is to use full user@domain names in /etc/dovecot/users and in the email-client “userid” field).

Now edit auth-passwdfile.conf.ext:

passdb {
  driver = passwd-file
  args = ... username_format=%u /etc/dovecot/users
}

userdb {
  driver = passwd-file
  args = username_format=%u /etc/dovecot/users
}

A Dovecot “passdb” indicates where (username -> password) mappings can be found to authenticate users over imap or sasl. A Dovecot “userdb” instead specifies where (username -> userinfo) mappings can be found; for some email-storage-mechanisms Dovecot needs to know a user’s “home dir” and UID in order to write files. This article actually sets up email storage in such a way that this info is irrelevant, but the userdb is still required. The passdb and userdb can be separate files, or can be a combined file in /etc/passwd-style layout.

The specified file /etc/dovecot/users should look somewhat like this:

# SHA512 passwords created via "sha512sum<enter>password<ctrl-d><ctrl-d>" then copy-and-paste
me:{SHA512}aabbccdd11223344....:1000:1000

Ensure the file has suitable access-rights! It must be readable by the user specified in “service auth” and in “service auth-worker” (10-master.conf) who is by default root. A safer config is to set these entries to user=$default_internal_user and ensure the file is readable by that user (dovecot).

As mentioned, this is basically /etc/passwd format with “columns” holding username, passwd, userid, groupid, description, shell, home-dir.

However the “passwd” column holds a password-hash prefixed with the hash-method used, so that Dovecot can hash an incoming password and compare it to the expected value.

When Dovecot is configured to deliver emails to a user’s homedir then the homedir is of course relevant; however in this article Dovecot is instead configured to store email under /var/dovemail/{user} so the homedir in this file is not actually relevant. Neither is description or shell, so only 4 columns are needed.

If you have problems with authentication, try editing /etc/dovecot/conf.d/10-logging.conf and setting the “verbose” options to “yes” to get more info in /var/log/mail.log.

By default, unencrypted-logins are disabled, which is what we want.

Dovecot Delivery

Edit file conf.d/10-mail.conf to specify where email is to be stored in the local filesystem. Set

mail_location = maildir:/var/dovemail/%u/Maildir
mail_access_groups=dovecot

Also create directory /var/dovemail, set its group to the dovecot user, and set the group-sticky-bit:

mkdir /var/dovemail
chgrp dovecot /var/dovemail
chmod g+s /var/dovemail

This will store user-specific settings for user me under /var/dovemail/me, and emails under /var/dovemail/me/Maildir. The “home” column of the userdb (file /etc/dovecot/users) will be ignored (and can be empty).

The files will be owned by the UID specified in the Dovecot userdb. When accessing emails on behalf of a user, Dovecot will “switch user” to the UID specified in this file, in order to have the needed filesystem read/write access. An alternative is to define a new system user to “own all emails” (by convention, name=vmail) and specify this via config-setting mail_uid. The file defining users can then omit userids completely.

Dovecot Services

Edit 10-master.conf to define which ports/sockets Dovecot will listen on. Postfix will be set up to not deliver (write) emails itself but instead to pass them on to Dovecot for storage; the protocol between Postfix and Dovecot will be LMTP over a local filesystem socket, so configure Dovecot to create/listen on that socket:

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    user = postfix
    mode = 0600
  }
}

Postfix will also be configured to use Dovecot’s password-database to authenticate users who perform SMTP-AUTH with Postfix (ie local users wanting to send email over the Postfix submission-port). Postfix will communicate with Dovecot using the SASL protocol over a local filesystem socket, so configure Dovecot to listen on that:

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
  }
}

See this Dovecot Wiki page for more information on Postfix+Dovecot+LMTP.

SSL

Connections from email clients to Dovecot’s IMAP port should always be encrypted. That means Dovecot needs to be given access to the SSL server-certificate for mail.example.com.

I presume you’ve already created such a certificate, so now edit conf.d/10-ssl.conf:

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.example.com/privkey.pem

Other

Edit conf.d/15-lda.conf and set the postmaster_address to postmaster@example.com.

Starting Dovecot

Just run “systemctl restart dovecot” (or service dovecot restart) to pick up all the changes. Check /var/log/mail.err and /var/log/mail.log for any error-messages.

Note: In order to start dovecot, postfix needs to have been installed (see next section) - though it doesn’t need to be configured or running.

Testing Dovecot

From the Dovecot wiki:

  openssl s_client -connect mail.example.com:993
  ==> "OK DOVECOT READY"
  A1 LOGIN username password
  ==> LOGGED IN
  A2 LIST "" "*"
  A3 EXAMINE INBOX

Other options:

  • doveadm user someuser checks whether user-lookup of the specified user works
  • doveconf userdb displays the current userdb configuration
  • use sendmail (after Postfix is configured) - which will place email directly on mailqueue bypassing smtpd.

See /var/log/mail for messages.

Install Postfix

Postfix can be simply installed via:

  • apt install postfix

Also install the SPF module for Postfix to verify SPF for incoming mail:

  • apt install postfix-policyd-spf-python

Set up Submission Port

Edit /etc/postfix/master.cf to add an extra SMTPD server specifically for outgoing email:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_client_restrictions=$mua_client_restrictions
  -o smtpd_helo_restrictions=$mua_helo_restrictions
  -o smtpd_sender_restrictions=$mua_sender_restrictions
  -o smtpd_recipient_restrictions=$mua_recipient_restrictions
  -o smtpd_relay_restrictions=$mua_relay_restrictions
  -o milter_macro_daemon_name=ORIGINATING

Don’t be worried about the many other “services” defined in this file; they don’t “run” unless needed - and our configuration will never need them.

The meaning of the entries here is described in the Postfix email theory article.

Note option -v to smtpd above; that turns on verbose logging in the standard syslogs (/var/log/syslog); remove that once everything is working well.

Here’s my full Postfix master.cf file:

# standard services
pickup    unix  n       -       y       60      1       pickup
cleanup   unix  n       -       y       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       y       1000?   1       tlsmgr
rewrite   unix  -       -       y       -       -       trivial-rewrite
bounce    unix  -       -       y       -       0       bounce
defer     unix  -       -       y       -       0       bounce
trace     unix  -       -       y       -       0       bounce
verify    unix  -       -       y       -       1       verify
flush     unix  n       -       y       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
showq     unix  n       -       y       -       -       showq
error     unix  -       -       y       -       -       error
retry     unix  -       -       y       -       -       error
discard   unix  -       -       y       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       y       -       -       lmtp
anvil     unix  -       -       y       -       1       anvil
scache    unix  -       -       y       -       1       scache
postlog   unix-dgram n  -       n       -       1       postlogd

# incoming from processes on same host
smtp      unix  -       -       y       -       -       smtp

# incoming from external systems (email to be delivered to Dovecot)
smtp      inet  n       -       y       -       5       smtpd

# incoming from registered users (email to be queued for sending)
submission inet n       -       y       -       2       smtpd -v
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_client_restrictions=$mua_client_restrictions
  -o smtpd_helo_restrictions=$mua_helo_restrictions
  -o smtpd_sender_restrictions=$mua_sender_restrictions
  -o smtpd_recipient_restrictions=$mua_recipient_restrictions
  -o smtpd_relay_restrictions=$mua_relay_restrictions
  -o milter_macro_daemon_name=ORIGINATING

# discussed later in section on SPF
policy-spf  unix   -     n       n       -       -       spawn
  user=nobody
  argv=/usr/bin/policyd-spf

Local delivery vs Virtual delivery

As described in the “Postfix Theory” article, Postfix smtpd first categorizes incoming email into:

  • “local” (when the recipient address domain is in $mydestination)
  • “virtual” (when the recipient address domain is in $virtual)

For emails with class=local, it then checks $local_recipient_maps to see if the recipient user exists - which looks in /etc/passwd and /etc/aliases by default. Assuming that passes, the emails are then delivered using the service specified in $local_transport; that is mapped in master.cf to an actual application to execute - usually /usr/lib/postfix/sbin/local. That then looks up the user in /etc/passwd again then usually writes emails directly to a file, although $mailbox_transport can be used to pass the email elsewhere, eg to Dovecot.

A similar process is applied to incoming emails with class=virtual, except that custom files are used instead of /usr/passwd, and that the “email domain” remains attached to the “userid”, ie virtual-delivery treats the recipient as “user1@example.com”, not as “user1”.

For a simple email-server handling just one email-domain and with just a few users, either approach works. Using the local approach does require each email-recipient to have a native unix account on the mailserver, but the account does not need to have login rights and for just a few users that is no great problem. And it allows per-user “forwarding files”, ie each user can determine if/where their email gets forwarded to. I initially set up my Postfix configuration with my target email-domain in $mydestination (thus categorising incoming emails as class=local), together with configuring $mailbox_transport to use LMTP to forward emails to Dovecot. However there is one disadvantage: setting up a catch-all email account (ie one to which email for unknown-users is saved) is very difficult (maybe impossible); the local delivery agent supports $luser_relay for this use-case, but that does not work in combination with LMTP, and I could not figure out how to configure catch-all behaviour on the Dovecot side.

I therefore use virtual delivery in this solution, even though I support only one email domain. Postfix users are defined in a plain text file (/etc/postfix/virtual) and Dovecot users are defined in a separate plain text file (/etc/dovecot/users). Having duplicated configuration is a shame, but the required file formats are different. Given a small number of users, this duplication is acceptable. Local delivery also requires duplicated entries (as far as I can tell). Having just a single definition for users between Postfix and Dovecot appears to only be possible when using a SQL database (both Postfix and Dovecot can then be configured to use the same SQL tables). Fortunately, setting up a “catch-all” account for Postfix virtual addresses is trivial, as is setting up email-forwarding (although changing the forwarding-address must be done by the mail-admin, not an end-user).

Configure Postfix main.cf

Postfix is a set of about a dozen separate applications. Each application has a set of variables that influence how it behaves; these variables have builtin defaults (usually very sensible ones) which can be overridden in file main.cf and those can be overridden via “-o” options on the commandline (as shown above).

So now edit /etc/postfix/main.cf as follows:

# Postfix Config settings for services started in master.cf
# See /usr/share/postfix/main.cf.dist for a commented, more complete version
#
# Note that these are _defaults_ for services, which can be overridden on the commandline specified in master.cf.
# In fact, these settings are simply variable-declarations that are referenced from elsewhere (typically master.cf)

# Indicate that this config-file is version-3.6-format, and that backwards-compat settings
# for earlier config-formats are not wanted
compatibility_level=3.6

# Basic settings
#
# Note that when $mydestination includes "example.com" then incoming mail for user@example.com will be considered "local".
mydomain = example.com
myhostname=mail.example.com
myorigin = $myhostname
mydestination = $myhostname, $myorigin, localhost, localhost.localdomain
mynetworks_style = host
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
append_dot_mydomain = no

# Some generic settings
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
readme_directory = no

# Send outgoing email direct to destination, not via some intermediate
relayhost =

# Rate Limiting
smtpd_recipient_limit = 5
smtpd_client_recipient_rate_limit = 50

# TLS parameters
# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.
smtpd_tls_security_level=may
smtpd_tls_cert_file=/etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache

smtp_tls_security_level=may
smtp_tls_CApath=/etc/ssl/certs
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_protocols=!SSLv2, !SSLv3

# Dovecot settings
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

# Handling incoming LOCAL email
#
# Aliases are transformed via alias_maps. Then the user is expected to exist in /etc/passwd (else email rejected).
# Then the email is accepted (sender is given an accepted-code) and the "local" delivery agent is applied. This
# delivery agent looks for ".forward" files, then uses local_transport
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases

# Handling incoming VIRTUAL mail
#
# Email to any address of form @foo where foo is in $virtual_alias_domains will be delivered via $virtual_transport
#
# Use service lmtp (from master.cf) for "virtual transport", passing that application/service
# some extra parameters ("unix:private/dovecot-lmtp") that indicate which socket to pass data over.
#
# Note that Postfix provides a program called "the virtual delivery agent" but that is not being used here; instead
# we deliver mail via LMTP to dovecot rather than letting Postfix deliver it directly to the local filesystem. That
# allows dovecot to consistently manage storing incoming mail along with POP/IMAP access to the mail. Unfortunately
# it is currently still necessary for postfix to know whether the target user exists before it does LMTP.
virtual_alias_domains = example.com
virtual_alias_maps = hash:/etc/postfix/virtual
virtual_transport = lmtp:unix:private/dovecot-lmtp
mailbox_transport = lmtp:unix:private/dovecot-lmtp

# == Custom variables used by the "submission" smtpd instance (see master.cf)
# ie "$mua*" are variables applied to connections to port 587 (mail user agents aka mail clients)
mua_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject
mua_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject
mua_client_restrictions =
mua_helo_restrictions = 

# == Custom variables used by the "primary" smtpd instance (see master.cf)
# ie variables used (implicitly) when receiving inbound email from other servers on port 25

# client-restrictions are rules applied to the (identity of the) mailserver sending mail (its IP address),
# ie very early in processing an incoming request.
#
# See https://docs.spamhaus.com/datasets/docs/source/40-real-world-usage/MTAs/020-Postfix.html
# A DQS_Key can be obtained by registering at spamhaus.com, then going to user portal
dqs_key = YOUR_SPAMHAUS_DQS_KEY
rbl_reply_maps = hash:$config_directory/dnsbl-reply-map
smtpd_client_restrictions =
  reject_rbl_client $dqs_key.zen.dq.spamhaus.net=127.0.0.[2..11],
  reject_rhsbl_sender $dqs_key.dbl.dq.spamhaus.net=127.0.1.[2..99],
  reject_rhsbl_helo $dqs_key.dbl.dq.spamhaus.net=127.0.1.[2..99],
  reject_rhsbl_reverse_client $dqs_key.dbl.dq.spamhaus.net=127.0.1.[2..99],
  reject_rhsbl_sender $dqs_key.zrd.dq.spamhaus.net=127.0.2.[2..24],
  reject_rhsbl_helo $dqs_key.zrd.dq.spamhaus.net=127.0.2.[2..24],
  reject_rhsbl_reverse_client $dqs_key.zrd.dq.spamhaus.net=127.0.2.[2..24]

# sender-restrictions are rules applied to the MAIL-FROM header (ie earlyish in processing)
# Support blocking annoying senders (aka "killfile")...
smtpd_sender_restrictions =
  check_sender_access hash:/etc/postfix/sender_blocklist

# relay-restictions are rules applied after all mail headers have been received (but before body).
# This allows validation of the "To:" header etc. The recipient-restrictions are also run
# in the same "phase" and the two rulesets _can_ be combined - but it's cleaner to separate them,
# and safer if recipient-restrictions contain any rules which may trigger "allow/permit".
# Note that incoming email is accepted only if it passes BOTH of smtpd_relay_restrictions AND smtpd_recipient_restrictions
#
# Rule reject_unauth_destination is what prevents this from being an "open relay".
relay_domains =
smtpd_relay_restrictions = reject_unauth_destination

# recipient-restrictions are rules applied after all mail headers have been received (but before body).
# This allows validation of the "To:" header etc.
smtpd_recipient_restrictions =
  reject_unauth_pipelining, reject_non_fqdn_recipient, reject_unknown_recipient_domain,
  check_policy_service unix:private/policy-spf

# Set max message size to 20MB (overriding default of 10MB)
message_size_limit = 20480000

# SPF settings
policy-spf_time_limit = 3600s

# Useful in initial setup
# defer_transports=smtp

There is a lot to explain here…

Most of these settings are for use by the SMTPD daemon, although some are also used by the Postfix-variant of sendmail which is used by Spamassassin among other things.

Variable mydomain should be your email domain, ie the bit after @ in email addresses intended for your users. The default is to use $myhostname without the first part of the name, eg when myhostname=somehost.example.com then implicitly mydomain=example.com. However it seems safer to me to define this explicitly.

Variable myorigin is (according to the docs) “the domain name that locally-posted mail appears to come from and that locally posted mail is delivered to”. On Debian/Ubuntu, this is set by default via file /etc/mailname while in others it defaults to $hostname. However I think it is clearer to define this explicitly. Warning: this is an email domain name for internal (local) delivery; if this value is equal to $mydomain but you are trying to use virtual delivery (as this article does) then incoming email for user1@$mydomain gets rejected with the very misleading message “User unknown in virtual alias table”.

Variable mydestination has been set up so email to user1@localhost or user1@$myhostname is still delivered as a “local” email (ie assumes there is a native unix account named user1). However this variable does not include $mydomain so such emails will not be classified as local.

The relay options are set to values that effectively disable “explicit relaying”. Here, the word relay means relaying between cooperating email servers, eg within a single company. We are setting up a personal email-server here, so this kind of “relay” is not needed. Later, the word relay is used to mean sending email to arbitrary email-servers on the internet; this reuse of terminology is unfortunate and confusing.

The SSL entries are hopefully obvious. Inbound email coming from external email-servers should really be transferred over an encrypted link for privacy, New emails being submitted by local users (via email client apps) over SMTP should definitely be encrypted to hide the transfer of passwords etc. And this encryption requires Postfix to have access to an SSL server certificate for mail.example.com. Outbound email from from this server to external email-servers should of course also be encrypted, but in that case the certificate is the responsibility of the remote server. Note that the “submission” SMTP server may require SSL (and we do set this up via a “-o” option in master.cf), but the SMTPD service on port 25 is not supposed to require SSL, just to offer it as an option.

The standard aliases are applied to emails of class “local”.

The email domain example.com (ie emails like user1@example.com) is defined as a recognised virtual domain. The valid users for this domain are defined in file /etc/postfix/virtual. File virtual_alias_maps defines not only which users exist, but also optionally defines “forwarding rules” equivalent to the “dot-forward” feature of the local-delivery-agent, eg forwarding all emails for a user to user@gmail.com. And the file can also define a “catch-all” target email address. As noted earlier, there is duplication between this file and /etc/dovecot/users that is unfortunately unavoidable AFAICT; there is no common file-format supported by both Dovecot and Postfix. Unified data is possible via an SQL database, but that is not worth-while for a small email server.

Setting virtual_transport indicates that emails for example.com should be passed via lmtp to Dovecot.

The config-item “defer_transports=smtp” can be useful when trying to get Postfix set up; it ensures that all email queued for the smtp-service to send out over the internet will instead be placed “on hold” until explicitly sent via postqueue -f to “flush queues”. It is then possible to check the deferred queue first before flushing the queues, to ensure no spammer has managed to route emails via your Postfix instance due to incorrect Postfix rules. Once your email server is registered in DNS (see later), spammers will start probing it within minutes. If your server is temporarily misconfigured, and starts forwarding such spam mails, then you can quickly land on a black-list from which it is difficult to get removed again. So blocking outgoing email can be a nice initial safety-net.

Now we come to potentially the most confusing part: mua-restrictions, client-restrictions, relay-restrictions, and recipient-restrictions.

Client-restrictions (*_client_restrictions) are used to reject entire remote mailservers, ie prevent a mailserver from submitting any mail. Here we reject any remote email-server which is on a spamhaus blocklist.

Recipient-restrictions (*_recipient_restrictions) are used to reject specific emails on the basis of the recipient address (“to-address”) - or any other header. For the smtpd instance that handles incoming email, we expect only email to someone@example.com; constraint reject_non_fqdn_recipient ensures the address does specify a domain, and reject_unknown_recipient_domain ensures the domain is example.com.

Relay-restrictions (*_relay_restrictions) are also used to reject specific emails on the basis of the recipient address; an email will only be accepted by smtpd if it passes the tests in both lists (see article Postfix Theory for why two lists are a good idea). There is usually just one rule in smptd_relay_restrictions: reject_unauth_domain which checks whether the domain of the to-address (eg example.com) is the local-domain or in var relay_hosts. And relay_hosts is usually empty (except for large orgs) - so the result is that this server cannot be abused by spammers to send email to arbitrary servers on their behalf (an “open relay”). Note that “auth” here means “allowed in config file”, not “is user authenticated”. For the “submission” smtpd instance which handles outgoing email from email clients, master.cf specifies mua_recipient_restrictions and the restrictions defined in that variable do allow specifying remote domains for the “to” address - but only after a “user login via sasl” (see sasl later).

Phew - done. Now emails are rejected on port 25 unless destined for a user defined in the virtual_alias_maps, and rejected on port 587 unless coming from a logged-in user.

Note on SASL Authentication

An email-server should be very careful about sending email into the internet unless it knows that the email is from a “trustworthy source”. Applications running on the local host are generally considered “sufficiently trustworthy”, as are applications which authenticate themselves via (username, password). Postfix thus supports authentication - but Dovecot also needs to authenticate users who want to view email via IMAP, and it is not good to have separate (user,pwd) databases for each. There are several possible solutions, but the most common is to make Dovecot the master “authenticator” and for Dovecot to offer authentication as a service on a local filesystem socket. Postfix can then be configured to pass on (user, hash) pairs to this socket, and read the ok/fail response back. Actually, the SASL protocol is more complicated than that, but the details are not relevant here. Note that checking which users exist is needed for incoming email, and this is not checked over SASL - only logins performed by users wanting to submit outgoing email. This can lead to some duplicated config - but for a few users the nuisance is bearable.

Restart Postfix

Run service postfix restart to pick up all changes. Check /var/log/mail.err and /var/log/mail.log for messages. If you have problems, any application listed in master.cf can have “-v” added to its options to increase the amount of logging it does (as shown above for the submission sevice).

Define Email Accounts

Define users in /etc/dovecot/users and in /etc/postfix/virtual as appropriate.

Note that a “local aliases” file has lines of form “aliasname:realname” while “virtual aliases” files have lines of form “aliasaddress <whitespace> realaddress”.

After updating /etc/postfix/virtual, run postmap /etc/postfix/virtual to generate file /etc/postfix/virtual.db.

Define “Sender Blocklist”

Create file /etc/postfix/sender_blocklist with content of form:

email-address   REJECT  "Spam"

where email-address can be a full address or a domain (example.com). Any spammers or other annoying email sources can be added to this file.

Run postmap /etc/postfix/sender_blocklist to convert this to “database form” (creates a file with suffix .db).

See Postfix docs for access databases for more details.

Testing

You should now be able to use a desktop email client to login-in to the submission port, set up encryption, and submit an email from foo@example.com to foo@example.com. After a few seconds, the email should appear in the IMAP inbox for the same account. Aliases should also work.

However before sending email to other hosts, DNS should be set up properly.

Useful Postfix Commands

Command postsuper can perform a range of useful functions, eg deleting all mail from a queue. Can only be executed by root. Examples:

  • postsuper -d ALL – delete all queued emails

Command postqueue can perform some other useful functions, and can be run by a regular user.

Command postconf shows the current configuration (and with the right flag, the default configuration). Examples:

  • postconf -d – displays default settings for all variables
  • postconf -n – displays all variables which differ from the default value (ie which have been overridden locally)

Command postfix can also perform some useful functions, eg:

  • postfix flush – send all queued mail now

Configure DNS

Now we need to inform other systems that email should be directed to this server, and set up some safety-nets to prevent other servers from impersonating our domain when sending spam.

  • ensure an A-record maps somehost.example.com->ip4-address
  • ensure an AAAA-record maps somehost.example.com->ip6-address
  • ensure a PTR record exists for somehost.example.com (ie maps ip4-address back to somehost.example.com)
  • define TXT records for SPF and DMARC (and optionally DKIM)
  • define CNAME record for mail.example.com -> somehost.example.com
  • define MX record for example.com -> mail.example.com

The first step is to ensure that the domain-name-registrar through which the base domain-name (example.com) is registered has registered NS records for the domain which point at a set of DNS servers that can be further configured. Some registrars also provide DNS servers and a suitable web-page. Virtual-hosting companies also do. In my case, the mail-server runs on a virtual server rented from digitalocean.com, and digitalocean.com provide DNS servers for their hosted systems; I therefore use my domain-name-registrar admin page to register NS records that point at the digitalocean domain-name servers and then register the above PTR/TXT/CNAME/MX records via an admin-page for the digitalocean DNS.

The A-record is the “standard” record for DNS; when somebody types http://somehost.example.com into a browser address-bar, or runs ping somehost.example.com then the A-record or AAAA-record is retrieved for that name to find the “real” address.

The PTR record is used by email-servers to filter out “real servers” from hacked desktops, routers, or other systems which have been taken over and used to generate spam; such things have an IP-address but won’t have a PTR record. The incoming ip-address is always available to the receiving email-server, so it can use that address as the key for a DNS-lookup of the PTR record. If no record exists, that is a strong indication that the remote system is not a “serious” server. The resulting server-name can also be used when verifying SPF records.

Each email has two “source addresses”: MAIL-FROM aka Return-Path, and the “From:” header. Spammers have a nasty habit of using the addresses of real (innocent) people in both of these addresses when generating spam. Many email servers therefore implement the Sender Policy Framework (SPF); for each email they take the supposed MAIL-FROM address (eg sender@example.com) and extract the domain (example.com) then perform a lookup of TXT records that have that key. If a TXT record is found which looks like an SPF-record then the data in that record will hold the host-name of all servers (usually one) that are permitted to generate emails with that address. This is then compared with the hostname found via the PTR-record lookup; a mismatch means the remote system is using faked email addresses and they are rejected. In other words, once you have registered an SPF record for yourmaildomain->yourmailhost then no spammer can generate spam with fake MAIL-FROM addresses that claim to be from your email-domain. And that can help keep your system off blackhole-lists. The exact syntax for SPF records is non-trivial, but the following is an initial guide:

v=spf1 +mx +a ip4:111.222.333.444 -all

Unfortunately, SPF does nothing to prevent spammers from faking the “From:” header - which is what the user actually sees. SPF is therefore a partial but not complete fix. Publishing a DMARC record in addition to the SPF record protects the from-header too (for all email servers that actually apply DMARC checks to incoming mail). However enabling DMARC is not addressed in this article.

The CNAME record is useful here because I want to pay for just one server which not only runs email but also hosts a website and various other stuff. However SSL certificates are issued for a specific hostname (aliases are possible, but tricky and letsencrypt doesn’t support them). A CNAME record is an elegant solution - it gives a dedicated hostname (eg mail.example.com) but points to the shared server. Registering an additional A-record is not so useful because digitalocean will only register a PTR record for the “real” hostname of a virtual server. The major email servers are fortunately smart enough to realize that MX:example.com->CNAME:mail.example.com->A:somehost.example.com->PTR:somehost.example.com is a valid setup, and don’t require the PTR record to map back to mail.example.com.

And the MX record says that email destined for “@example.com” should be sent over a socket opened to mail.example.com - ie maps between two different “namespaces” (mail-namespace to host-namespace).

There is yet another framework designed to frustrate spammers - DKIM. It has roughly the same effect as SPF, though via a different approach. Hopefully I can cover that in another article in the future. However it doesn’t seem critical to implement that straight away; I’ve had no problems without it so far.

Configure the Email Client

The client should read email using IMAP on port 993.

The client should send email using SMTP on port 587 (the submission port). Trying to send emails to external addresses via port 25 will just trigger “relaying not permitted” which is exactly correct; a system which accepts emails on port 25 (which does not require user authentication) and forwards them to external systems is called an “open relay”. That would be very bad for us, very good for the spammers.

In both cases, the username to login with is without domain (assuming you configured auth_username_format = %Ln in Dovecot, as recommended), and the password is whatever was entered in /etc/dovecot/users.

Note that some systems don’t have a “submission” instance, and instead carefully order the validation-constraints for the main SMTPD instance so that authenticated remote senders have different rights to unauthenticated ones. However the rules need to be exactly right or an “open relay” can accidentally be created. Separating “authenticated remote senders” onto their own port (the “submission port”) makes the rules easier to define and less vulnerable to mistakes.

SpamAssassin

Spamassassin can be installed with:

  • apt install spamassassin spamc
  • systemctl enable --now spamassassin.service

Then update /etc/postfix/master.cf:

smtp      inet  n       -       y       -       5       smtpd
  -o content_filter=spamassassin

spamassassin unix  -     n       n       -       -       pipe
  user=debian-spamd
  argv=/usr/bin/spamc -f
  -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Spamassassin configuration is also discussed in article Postfix Theory.

Spamassassin simply assigns a “spam score” to each email, and stores it into an email header; if you wish to take some action based upon that score then one option is to configure Dovecot to use sieve rules. Postfix itself does not do anything with the information that spamassassin adds to the email.

Spamassassin scores email based upon a set of rules which are initially installed into /usr/share/spamassassin. When installing spamassassin on Ubuntu via apt then /etc/cron.daily/spamassassin is also installed to ensure these rules are kept up-to-date via the sa-update tool. Somewhat confusingly, the updated rules are not written into /usr/share/spamassassin but into /var/lib/spamassassin/3.004001 instead. They are then “compiled” into /var/lib/spamassassin/compiled; presumably this is the same location that the original rules in /usr/share/spamassassin are compiled into, and thus spamd (the spamassassin daemon) sees the updated rules.

IMPORTANT: by default, /etc/cron.daily/spamassassin is disabled; you need to edit it and set CRON=1 for updates to actually work.

The spamassassin wiki has further information on the sa-update tool. Unfortunately, at the current time the instructions on the wiki refer to updates.spamassassin.org which no longer exists; the spamassassin project recently moved to the Apache foundation, and it appears that some things have changed but not yet been updated in the wiki. By the way, there appears to be a bug in the /etc/default/spamassassin file used by /etc/cron.daily/spamassassin: it sets PIDFILE=/var/run/spamd.pid but the systemd spamassassin file writes the pid to /var/run/spamassassin.pid. However the cron-script does not use this variable.

Enabling SPF Checks

Publishing an SPF record as described earlier in the section on DNS protects your domain - ie prevents spammers from faking your domain when sending email to mail-servers that perform SPF checks. That’s critical to prevent your domain from being black-listed as a “spam origin”.

Your Postfix instance should also perform SPF checks on incoming email so spam using faked addresses is blocked before delivery to your users. The necessary setup has already been described above. See article Email Validation on this site for more details.

Enabling DMARC Checks

Ideally, every email server should have DMARC validation enabled; this extracts the “domain” part of the “From:” header of each email, and looks for a DMARC record in DNS matching that domain. If one is present, then either an SPF record should exist for that domain, or the email should be signed via DKIM.

While enabling DMARC is not particularly difficult, it is not built-in in Postgres. You can find futher information in a separate article on this site.

Configuring New Users

Just to repeat for clarity: with the setup described above, there are two sources of user information: Postfix and Dovecot. The process of adding a new user is therefore:

  • edit /etc/dovecot/users to add the username and password
  • edit /etc/postfix/virtual to add the user then execute postmap virtual to generate file virtual.db

Other Notes

I found that as soon as I registered an MX record in DNS, the number of login-attempts via SSH as user root increased. It is therefore probably worth checking that you have login-as-root disabled in ssh (ie login as another user, then su to root): file /etc/ssh/sshd_config should contain PermitRootLogin no.

Setting up fail2ban is also recommended.

References