IT Notes - e-mail

Make your own E-Mail server - Part 2 - Adding Webmail and More with Nextcloud

Stefano Marinelli — Thu, 21 Mar 2024 07:15:00 +0000

In the first part of this series, I detailed how to install and configure a complete mail server within a FreeBSD jail. While accessible via IMAP and SMTP, I intentionally did not cover installing a webmail interface.

Installing a "simple" webmail is not complex, but nowadays, we expect more integrated solutions, including contact books (synchronizable with computers and mobile devices) and calendars. For this reason, in this second part, I will be utilizing Nextcloud.

Nextcloud represents a comprehensive groupware solution today, extendable through "apps" that can be installed directly from the administration menu. This setup offers numerous advantages, including authenticating Nextcloud users through the IMAP server (thus, a mail server user will automatically have a Nextcloud account, and password changes on the mail will reflect on Nextcloud too), the ability to use integrated Contacts (also in webmail) and Calendars, and the option to attach or save attachments directly in Nextcloud.

We will use the reverse proxy configured in part 1, which will "pass" connections to Nextcloud. First, a domain name must be defined, which will be used to connect to the server. In this example, I will use "nextcloud.example.com."

To do this, enter the proxy's jail:

bastille console nginx-proxy

Modify the file /usr/local/etc/nginx/nginx.conf, adjust the worker_processes (so nginx will start a number of processes equal to the number of cores on the machine it's running on), and create the virtual host for Nextcloud:

worker_processes auto;

In the http block, change the maximum file size limit and create the virtual host on http - we will generate the https block and related certificates later:

client_max_body_size 0;

server {
    listen 80;
    server_name nextcloud.example.com;

    location / {
        proxy_buffering off;
        proxy_http_version 1.1;
        fastcgi_hide_header X-Powered-By;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto https;
        proxy_pass https://192.168.123.5;
        proxy_connect_timeout   10m;
        proxy_send_timeout      10m;
        proxy_read_timeout      10m;
    }

    location /.well-known/carddav {
        return 301 $scheme://$host/remote.php/dav;
    }

    location /.well-known/caldav {
        return 301 $scheme://$host/remote.php/dav;
    }
}

The proxy_pass is in https because, later on, Nextcloud will respond in https. Although not necessary, this approach is simpler given the method used to install Nextcloud. Moreover, it allows for the reverse proxy to be on a different server than Nextcloud, without the need for a VPN.

Now, enable and start nginx:

service nginx enable
service nginx start

After setting the listener to port 80, it's now appropriate to generate the certificate.

A simple command will request the certificate and modify the nginx configuration to accept connections over https:

certbot --nginx -d nextcloud.example.com

Upon completion of this command, the virtual host for https will be correctly generated, and an automatic redirect will be set up for all connections arriving on http to be "moved" to https.

Exit the jail, and it's time to create the jail for Nextcloud.

bastille create -B nc 14.2-RELEASE 192.168.123.5/24 bridge0

Enter the jail and add the gateway in rc.conf:

defaultrouter="192.168.123.1"

After restarting the jail, it will now be possible to install Nextcloud.

FreeBSD has a fairly updated package, but when it comes to web applications, I prefer to have more control. Usually, I install everything manually, but for Nextcloud, I found an interesting script made by Gibran Khan that takes care of everything, i.e., installing Apache, PHP, ClamAV, all dependencies. It will also configure Nextcloud and make it (almost) ready to use.

Since Gibran's script was structured for version 27, used PHP 8.2, and had other small things that were not perfect for my use, I decided to modify it slightly, mainly to support Nextcloud 28 and PHP 8.3. Therefore, the description that follows will be related to my fork available here: https://github.com/draga79/NextCloudOnFreeBSD

So, enter the "nc" jail, download the script, and start with the configuration:

bastille console nc
fetch https://github.com/draga79/NextCloudOnFreeBSD/archive/refs/heads/release.zip
unzip release.zip
cd NextCloudOnFreeBSD-release
./pre-install.sh

A configuration file will be generated that we need to modify. The file is install.conf.

You will certainly need to change the HOST_NAME (inserting the one you have chosen, in this case, "nextcloud.example.com") and, unless you are in Italy, I would also suggest changing the COUNTRY_CODE and TIME_ZONE.

Once completed, you can proceed with the actual installation:

./install.sh

At the end of the installation, the login data will appear (which will also be saved in a file named nextcloud.example.com_reference.txt). The installer will also set up a cron job executed by the user "www", which will handle, every 5 minutes, the background operations required for Nextcloud to function properly. In some cases, I've noticed that it's necessary to modify the cronjob, save it, and then it will start working correctly. If that's the case, simply open it with crontab -u www -e, save, and exit.

Now, modify the Nextcloud configuration file, /usr/local/www/nextcloud/config/config.php, and:

Replace overwrite.cli.url with your domain (https://nextcloud.example.com).
Add the proxy to the trusted proxies (if not present, insert the entire block):

 'trusted_proxies' =>
  array (
    0 => '192.168.123.2',
  ),

Modify the file /usr/local/etc/apache24/httpd.conf and change the Listen directive to:

Listen *:443

Restart Apache:

service apache24 restart

To simplify things, it's advisable to make a modification: in /etc/hosts, add a line to ensure the mail server can be reached via LAN through its domain name. Therefore, add the line:

192.168.123.3 mail.example.com

In this manner, when Nextcloud attempts to connect to the mail server, it will do so directly over the LAN. This direct connection is essential for the IMAP authentication plugin, as it verifies that the SSL certificate of the mail server matches the hostname being called.

Now log in to Nextcloud with the admin credentials that the installer provided you earlier. Nextcloud is now installed and initialized. We need to perform two operations - install the Snappymail app (if you want to use it, but generally, I recommend it over the integrated mail client in Nextcloud) and enable IMAP authentication to avoid having duplicate users (i.e., mail user and Nextcloud user).

Click on the menu at the top right (the big A) and go to Apps. Under "Integration," you will now be able to install "Snappymail" and "External User Authentication," which, however, is currently listed as "Untested" because it has been tested for a previous version of Nextcloud (but it works on version 28 too). You will have to click it twice, because the first click will only enable untested apps.

Once the apps are installed, go to "Administration Settings" and "Additional Settings," then to the Snappymail administration panel. Now add your domain, so click Add Domain. The domain name will be that of your emails (example.com). Therefore, in the IMAP server field, enter the name of the mail server (mail.example.com), and set the security to STARTTLS. Do the same for SMTP and SIEVE.

Perform the test (using the username and password of an existing and valid mailbox) and, once everything is validated, save.

You will now be able to log in into your webmail - in the top bar of Nextcloud there will be two icons, one for the integrated webmail client and the other for Snappymail. The integrated webmail client has some features that are not present in Snappymail (such as delayed sending of messages), so it may be useful to keep both active. Currently, Snappymail will require a username and password every time you connect (even if you are already logged into Nextcloud), but it's possible, in user settings, to enable automatic login once authenticated in Nextcloud.

Now is the time to configure IMAP authentication. Return to the Nextcloud jail and modify the Nextcloud configuration file, i.e., /usr/local/www/nextcloud/config/config.php. Add a block like this:

'user_backends' => array(
    array(
        'class' => '\OCA\UserExternal\IMAP',
        'arguments' => array(
            'mail.example.com', 993, 'ssl', 'example.com', false, false
        ),
    ),
),

'example.com' in the fourth field is used to limit the authentication scope to that domain. More information can be found on the official plugin page.

After saving, you will be able to log out as admin and log in with your mail account. Unless there are errors, you will be catapulted into your new user, created and authorized through IMAP authentication.

Congratulations, you are now able to use Nextcloud by logging in with your email credentials and also use it as a basis for webmail, leveraging the contacts module for the email address book as well.

Nextcloud can offer intriguing prospects for everyday use. In addition to its seamless integration with mobile devices, tools like 'Nextcloud Talk' and 'Note' will enable users to avoid the use of external solutions and maintain control over their data.

It will also be possible to synchronize contacts and calendars with mail clients (like Thunderbird) or the address books of various stationary and mobile devices. The procedure varies depending on the device used, but it's just a matter of synchronizing via CalDAV and CardDAV. Alternatively, you can set up Z-Push and use the Microsoft Exchange ActiveSync (EAS) protocol, but this will be described in part 3 of the series.

Addendum: Adding Antivirus Support to Spam Filtering

The automatic installation script, alongside Nextcloud, installed other components including ClamAV to check uploaded files in real time. We can leverage this installation (since ClamAV consumes significant resources, it makes no sense to install two instances) and integrate this instance of ClamAV with rspamd, to also perform antivirus checks on emails.

Modify /usr/local/etc/clamd.conf

Uncomment:

TCPSocket 3310

Restart clamd to listen on port 3310 as well:

service clamav-clamd restart

Return to the mail jail:

bastille console mail

Create the file /usr/local/etc/rspamd/local.d/antivirus.conf with the following content:

clamav {
  # If set, force this action if any virus is found (default unset: no action is forced)
  action = "reject";
  message = '${SCANNER}: virus found: "${VIRUS}"';
  # Scan mime_parts separately - otherwise, the complete mail will be transferred to AV Scanner
  #attachments_only = true; # Before 1.8.1
  #scan_mime_parts = true; # After 1.8.1
  # Scanning Text is suitable for some av scanner databases (e.g., Sanesecurity)
  #scan_text_mime = false; # 1.8.1 +
  #scan_image_mime = false; # 1.8.1 +
  # If `max_size` is set, messages > n bytes in size are not scanned
  #max_size = 20000000;
  # symbol to add (add it to metric if you want non-zero weight)
  symbol = "CLAM_VIRUS";
  # type of scanner: "clamav", "fprot", "sophos", or "savapi"
  type = "clamav";
  # If set true, a log message is emitted for clean messages
  #log_clean = false;
  # Prefix used for caching in Redis: scanner-specific defaults are used. If Redis is enabled and
  # multiple scanners of the same type are present, it's important to set prefix to something unique.
  #prefix = "rs_cl_";
  # For "savapi" you must also specify the following variable
  #product_id = 12345;
  # servers to query (if port is unspecified, scanner-specific default is used)
  # can be specified multiple times to pool servers
  # can be set to a path to a unix socket
  servers = "192.168.123.5:3310";
  # if `patterns` is specified virus name will be matched against provided regexes and the related
  # symbol will be yielded if a match is found. If no match is found, the default symbol is yielded.
  patterns {
    # symbol_name = "pattern";
    JUST_EICAR = '^Eicar-Test-Signature$';
  }
  # In version 1.7.0+ patterns could be extended
  #patterns = {SANE_MAL = 'Sanesecurity\.Malware\.*', CLAM_UNOFFICIAL = 'UNOFFICIAL$'};
  # `whitelist` points to a map of signature names. Hits on these signatures are ignored.
  whitelist = "/usr/local/etc/rspamd/local.d/antivirus.wl";
}

Restart rspamd in the “mail” jail and, from this point on, viruses will be automatically rejected.

Make your own E-Mail server - Part 1 - FreeBSD, OpenSMTPD, Rspamd and Dovecot included

Stefano Marinelli — Fri, 08 Mar 2024 07:15:00 +0000

The main power of the Internet has always been one: decentralization.

Ever since I've been able to, I've always managed my email boxes independently and provided mail hosting services to my clients. Over the years, things have become increasingly complex: on one hand, there's been a disproportionate increase in spam, and on the other, big players (like Google and Microsoft) have tried to gain a lot of ground in managing these services. Private users can obtain many free services, and business users no longer need to worry about the underlying infrastructure. However, in my opinion, the price to pay is very high: the loss of ownership of one's data.

These operators, in fact, use full access to our emails to improve their services or analyze us with the aim of selling us advertisements.

As often happens in these cases, many users appreciate this type of approach and have switched to the services of big players, causing a progressive increase in the level of influence that these companies can have on something free and decentralized like email.

In my opinion, it still makes sense to manage one's own mail servers. Standards evolve, and it is therefore appropriate to follow innovations, adapt to best practices, and secure one's services and users.

Over the last ten years, I have also used and installed various Zimbra OSE servers. Aside from a few minor issues, the setup has proven stable and reliable. Recently, the company that develops this excellent tool has decided to no longer provide packages for the Open Source version. They can be generated with scripts provided by some (great!) users, but they cannot be a solid and stable base in the long term.

I have therefore decided to gradually return to a modular, adaptable, customizable, and updatable mail host setup without worries or headaches. Such a system can include any kind of service (including webmail, integration with caldav/carddav, etc.) and it will be possible to disable what is not strictly necessary. Safer and more secure.

This will be the first in a series of articles, and at the end of this reading, you will have a secure, modern, reliable, and modular mail server. The instructions are designed for FreeBSD and related jails but with very few modifications can be applied to any BSD as well as Linux or other similar operating systems.

Setup Planning

Over the years, I've used many different SMTP servers. Originally, the good old Sendmail. Subsequently, and for many years, I used Exim, also because it was the default system in Debian. I then migrated almost all setups towards Postfix, probably the most widespread smtp server on the net, and I never had any particular problems. In recent years, I have decided to use, when possible, the excellent OpenSMTPD. Being based on OpenBSD and easily installable on other OSes, it shares the primary concepts of security and reliability of OpenBSD as well as the syntax (both in command line and in configuration files) of all other OpenBSD tools. For this type of setup, I will use opensmtpd.

The choice to use FreeBSD (rather than OpenBSD) stems from two main factors:

The possibility of dividing into jails, physically separating the various services and minimizing dependencies.
The possibility of using ZFS, with all the advantages it brings: the ability to take snapshots, the ability to perform backups quickly and efficiently, and the ability to migrate entire jails without particular problems.

The other tools that will be used are:

Rspamd - one of the best integrated systems for checking incoming mail. Customizable, adaptable, and modular, it also provides interesting monitoring systems on the type of incoming and outgoing mail.
Redis - which will be used by rspamd to manage its "memory".
Dovecot and Sieve - for delivering mail to local mailboxes as well as allowing remote email consultation (via imap) and enabling the setting of rules thanks to Pigeonhole Sieve.

The users of the mail server will not be "physical" users of the server itself, but virtual users, totally unrelated to system users.

This article will be long and detailed. The process may seem complex at first glance, but in reality, it's simpler than one might think. In the classic Unix philosophy, each component performs a task and does it well. Configuring the individual components separately will ensure greater scalability, reliability, and resilience to updates as well as simpler debugging compared to all-in-one solutions (like many of those found in convenient, ready-to-use docker-compose.yaml files).

Installation and Configuration

I won't describe the FreeBSD installation and configuration procedure. In my case, I activated a VPS on Hetzner - in this situation, the smallest of the available ARM servers. Yes, it's possible to efficiently manage a mail server for several users on a machine costing less than 4 euros a month. ZFS can be useful, but it's not necessary for this type of setup.

The first step is therefore to choose a location to install everything. The main condition is that the VPS/server/host must have at least one static public IP and, in 2024, I now consider it necessary to have (at least) one IPv6 address. They must also be able to send/receive mail on port 25, which is not guaranteed and not all VPS providers allow. Hetzner, for example, blocks port 25 for new users.

Once you have identified and acquired the server on which to install everything, it's appropriate to check that the IPs (both IPv4 and IPv6) to be used are not on any of the many blocklists/blacklists. My experience indicates that at least 60% of the IPs assigned to me are on one or more of these lists, so the first step is to request delisting, to ensure proper operation when the server is ready.

To Encrypt or Not to Encrypt?

A subsequent assessment is whether to use encryption for the data. Connections to and from the mail server will always be encrypted, but it's also possible to encrypt the data at rest. In the case of FreeBSD, it's possible to enable encryption for the entire installation (the entire virtual disk will be encrypted, and ZFS (or UFS) will be created on top of it) or, in the case of using ZFS, it will be possible to encrypt only the root dataset of the jails, the one that will be used by BastilleBSD. In the first case, it will be necessary to enter the password at every boot of FreeBSD, therefore connect via console at every restart and enter the password. In the second case, FreeBSD will be able to boot but it will be necessary to connect and enter the keys before mounting the BastilleBSD ZFS dataset. None of these solutions will prevent unauthorized access to emails in case of machine compromise. The main advantage will be "only" not to save the emails in clear on disks which, in the case of a VM, will still be shared and managed by third parties. In the case of physical servers, however, things will undoubtedly be different.

Encrypting the entire disk must be done directly during the FreeBSD installation phase, very simply. As for the option to encrypt only the BastilleBSD dataset, you can proceed later, with a command similar to:

zfs create -o encryption=on -o keylocation=prompt -o keyformat=passphrase zroot/bastille

at every reboot, it will be necessary to enter the password and mount the datasets:

zfs load-key -r zroot/bastille
zfs mount -a

Configuring FreeBSD and BastilleBSD

After the installation of the OS, the first step is to configure IPv6 on the VPS. In the case of Hetzner, unfortunately, they only provide a /64, so it will be necessary to segment the assigned network. In this example, it will be divided into /72 subnetworks - to find valid subclasses, it will be possible to use a calculator.

The /etc/rc.conf file should have entries similar to:

ifconfig_vtnet0="DHCP"
ifconfig_vtnet0_ipv6="inet6 2a01:4f8:cafe:cafe::1 prefixlen 72"
ipv6_defaultrouter="fe80::1%vtnet0"

In short, keep the base address assigned by Hetzner, but change the prefix length to 72 - thus giving the possibility of having other networks available.

It is now necessary to enable forwarding for IPv4 and IPv6. Add these lines to the /etc/sysctl.conf file:

net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1

After reboot, test it:

ping6 google.com

If everything has been configured correctly, the ping will be executed and google.com will reply.

It will also be necessary to set up reverse DNS, otherwise many mail servers will reject our emails. This must be done both for the IPv4 address and the designated IPv6 address (within the class - an operation that can also be done after creating the jails, but before starting to send email messages). Also, create the correct A and AAAA records on the DNS for the "mail.example.com" domain.

The FreeBSD installer does not automatically update the operating system, so it is appropriate to do so now:

freebsd-update fetch install

It seems, moreover, that there is still an old bug still present and manifesting when, on a FreeBSD server installed in a VM based on KVM (thus also those of Hetzner), routing is performed (as in our case) between VNET jails and host.

Adding this configuration to /boot/loader.conf will solve the problem:

hw.vtnet.X.tso_disable="1"
hw.vtnet.tso_disable="1"
hw.vtnet.lro_disable="1"
hw.vtnet.X.lro_disable="1"
hw.vtnet.csum_disable="1"
hw.vtnet.X.csum_disable="1"

It's time to install and configure BastilleBSD. This handy tool will make managing jails much simpler and more straightforward, and its configuration is well described on the project page.

Bridge and Firewall

For some time now, I've been favoring the use of VNET jails for these types of setups. I believe that having a complete network stack within the jails gives more freedom in configuration, firewall management (applicable also within individual jails), and technical possibilities (such as creating VPNs within the jails themselves, ensuring greater portability).

I suggest creating a bridge on the FreeBSD server and placing all the jails within this bridge. It will be enough to modify the /etc/rc.conf file and change/add:

cloned_interfaces="lo1 bridge0"
ifconfig_bridge0="inet 192.168.123.1 netmask 255.255.255.0"
ifconfig_bridge0_ipv6="inet6 2a01:4f8:cafe:cafe:100::1 prefixlen 72"
ipv6_gateway_enable="YES"
ipv6_activate_all_interfaces="YES"
gateway_enable="YES"

Now let's configure the firewall. In this example, I will use the IPv4 and IPv6 addresses that I will later assign to the jails. Here's an example of a working pf.conf:

ext_if="vtnet0"

set block-policy return
scrub in on $ext_if all fragment reassemble
set skip on lo
set skip on bridge0

table <jails> persist
# IPv4 private address ranges
table <private> const { 10/8, 172.16/12, 192.168/16 }
nat on $ext_if from 192.168.123.0/24 to ! <private> -> ($ext_if:0)
nat on $ext_if from <jails> to any -> ($ext_if:0)
rdr-anchor "rdr/*"
#rdr via ipv4 to mail
rdr pass on $ext_if proto tcp from any to ($ext_if) port { 25, 465, 587, 143, 993 } -> 192.168.123.3
#rdr via ipv4 to nginx-proxy
rdr pass on $ext_if proto tcp from any to ($ext_if) port { 80, 443 } -> 192.168.123.2

block in all
#PASS ICMP
pass in quick proto icmp from any to any
# Pass ICMP on ipv6
pass quick proto ipv6-icmp
pass out quick keep state
antispoof for $ext_if inet
pass in inet proto tcp from any to any port ssh flags S/SA keep state
#Pass ipv6 to mail jail
pass in quick on $ext_if inet6 proto tcp from any to 2a01:4f8:cafe:cafe:100::25 port { smtp, 465, 587, 143, 993 }
#Pass ipv6 to nginx-proxy jail
pass in quick on $ext_if inet6 proto tcp from any to 2a01:4f8:cafe:cafe:100::80 port { 80, 443 }

At this point, it is advisable to reboot the machine, to be sure that everything comes back up in the right way.

Creating the "nginx-proxy" and "redis" Jails

Let's start with the first jail. It will be an nginx reverse proxy. At the moment it will not be used as such but will be useful as a machine for generating certificates and, in the future, to act as a reverse proxy for various internal web services.

bastille create -B nginx-proxy 14.3-RELEASE 192.168.123.2/24 bridge0

Once the jail is created, it will be necessary to modify some configurations. In the jail itself (bastille console nginx-proxy), modify the /etc/rc.conf file and add the IPv4 gateway and IPv6 configurations, i.e., giving the address specified earlier in the pf.conf of the FreeBSD server and as the gateway the IP address of the related bridge:

defaultrouter="192.168.123.1"
ifconfig_vnet0_ipv6="inet6 2a01:4f8:cafe:cafe:100::80 prefixlen 72"
ipv6_defaultrouter="2a01:4f8:cafe:cafe:100::1"

Restart nginx-proxy with bastille restart nginx-proxy, and you will be able to re-enter the jail. At that point, ping google.com and ping6 google.com should work. The jail will then be able to operate, responding on 80 and 443 both in IPv4 (thanks to the NAT configured previously on the FreeBSD server) and in IPv6, being directly connected in routing.

Now install the necessary software:

pkg install -y nginx py311-certbot py311-certbot-nginx

To automatically renew the certificates, add this line to /etc/periodic.conf:

weekly_certbot_enable="YES"

It's now possible to generate the certificate, which we will also use for all other exposed services:

certbot certonly -d mail.example.com

Once the certificates are generated, we will need to make a small change as opensmtpd (which we will install later in another jail) is (rightly) very restrictive on permissions:

chmod -R 400 /usr/local/etc/letsencrypt/live/mail.example.com/

Now let's create a jail for redis. I usually put it in a dedicated jail also in terms of clustering/multi-server setup:

bastille create -B redis 14.3-RELEASE 192.168.123.4/24 bridge0

Also for the redis jail, it's appropriate to set up the gateway, so in the /etc/rc.conf of the jail:

defaultrouter="192.168.123.1"

I don't configure IPv6 as it's not necessary (this jail will only be reachable from the LAN) but it's possible to do so. Once the jail is restarted (bastille restart redis), it will be able to reach the outside:

pkg install -y redis

Not wanting to disable the protected mode of redis and not wanting to open it without authentication, it will be appropriate to modify the /usr/local/etc/redis.conf configuration file and add the password and some options to optimize its use with rspamd:

requirepass cafecafe
maxmemory 512mb
maxmemory-policy volatile-lru

Of course, set a unique password. Also, change the line concerning the bind:

bind *

Now execute:

service redis enable
service redis start

Redis should be active and ready to receive connections.

The "mail" Jail

It's time to create the jail with the main services:

bastille create -B mail 14.3-RELEASE 192.168.123.3/24 bridge0

and, also in this case, it will be necessary to enter the jail and modify some configurations in /etc/rc.conf:

defaultrouter="192.168.123.1"
ifconfig_vnet0_ipv6="inet6 2a01:4f8:cafe:cafe:100::25 prefixlen 72"
ipv6_defaultrouter="2a01:4f8:cafe:cafe:100::1"

Once the jail is restarted, it will be possible to install the necessary packages:

pkg install -y rspamd opensmtpd opensmtpd-filter-senderscore opensmtpd-filter-rspamd dovecot dovecot-pigeonhole

Please note: the opensmtpd-extras- packages are currently broken so we'll compile the table file support:

pkg install autoconf automake git-tiny
git clone https://github.com/OpenSMTPD/table-passwd.git
cd table-passwd
sh bootstrap
./configure
make
make install
cp table-passwd /usr/local/libexec/opensmtpd/
cp table-passwd.5 /usr/local/share/man/man5/

It is now appropriate to give this jail an FQDN, as it will be the name with which it presents itself when connecting to the outside. Just modify /etc/rc.conf:

hostname="mail.example.com"

The hostname should be equivalent to the reverse DNS set earlier, so the machine will present itself with a name that, doing a reverse lookup, will correspond to the IP of origin.

This jail does not contain the certificates that are present in the nginx-proxy jail. There are various methods to share them, such as using rsync, putting them on an NFS share, etc., but the simplest in this case will be to do a bind mount between the two jails, an operation that BastilleBSD can handle automatically.

Create the correct directory in the "mail" jail:

mkdir /usr/local/etc/letsencrypt

Exit the jail, shut it down, and modify the fstab file of the jail (e.g., /usr/local/bastille/jails/mail/fstab) by adding a line like this:

/usr/local/bastille/jails/nginx-proxy/root/usr/local/etc/letsencrypt /usr/local/bastille/jails/mail/root/usr/local/etc/letsencrypt nullfs ro 0 0

Start the jail, and at that point, the directory created earlier should display the directories and certificates from the nginx-proxy jail.

Due to spammers who, year after year, become increasingly aggressive, many mail servers require perfect configuration of the main sender authentication methodologies. Today, the main (and now necessary, under penalty of message non-delivery) are SPF, DKIM, and DMARC.

The first step is to generate a DKIM key for the domain(s) that will send mail from the server we are configuring. This operation can be performed in a few steps, by placing the keys in the /usr/local/etc/mail/dkim directory:

mkdir /usr/local/etc/mail/dkim
cd /usr/local/etc/mail/dkim
openssl genrsa -out example.com.key 2048
openssl rsa -in example.com.key -pubout -out public.key
chmod 0400 example.com.key
chown rspamd *

Since rspamd will take care of signing outgoing messages, it is appropriate that only the related user be able to read the file with the private key.

Once the pair of keys has been created, it will be necessary to configure DNS to provide the public key to mail servers that will request it. The record should look like this:

mail._domainkey   TXT   "v=DKIM1;k=rsa;p=your-public-key-goes-here"

Once the record has been entered and propagated, it will be possible to test its correctness through one of the many websites that offer this service, remembering that the selector we have configured is "mail" (the first part of the record). Obviously, this is a free text, which can be modified at will (but will then need to be specified in the rspamd configuration, later on).

Being in the DNS configuration phase, it will also be appropriate to create SPF and DMARC records:

SPF record:

example.com. IN TXT "v=spf1 a ip4:your.ip.address ip6:2a01:4f8:cafe:cafe:100::25 mx ~all"

DMARC:

_dmarc.example.com. IN TXT "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@example.com"

These records, obviously modified based on the chosen domain, are configured conservatively but suitable for a test. Once the entire setup is in place, it will be possible to handle things in a more restrictive manner.

The mail server users will be "virtual" users (i.e., not system users of the server), so all their mail will be "handled" by a unique user, which needs to be created:

pw user add vmail -u 2000 -d /var/vmail -s /usr/sbin/nologin
mkdir /var/vmail
chown vmail /var/vmail

Let's now create some files and directories, useful for the subsequent configuration:

touch /usr/local/etc/mail/passwd
touch /usr/local/etc/mail/virtuals
mkdir /usr/local/etc/rspamd/local.d

Remove the current /usr/local/etc/mail/smtpd.conf and replace it with content like this:

table passwd passwd:/usr/local/etc/mail/passwd
table virtuals file:/usr/local/etc/mail/virtuals

pki mail.example.com cert "/usr/local/etc/letsencrypt/live/mail.example.com/fullchain.pem"
pki mail.example.com key "/usr/local/etc/letsencrypt/live/mail.example.com/privkey.pem"

filter check_dyndns phase connect match rdns regex { '.*\.dyn\..*', '.*\.dsl\..*' } \
    disconnect "550 no residential connections - Thou shalt not pass"

filter check_rdns phase connect match !rdns \
    disconnect "550 no rdns - Thou shalt not pass"

filter check_fcrdns phase connect match !fcrdns \
    disconnect "550 no FCrDNS - Thou shalt not pass"

filter senderscore \
    proc-exec "/usr/local/libexec/opensmtpd/opensmtpd-filter-senderscore -blockBelow 10 -junkBelow 70 -slowFactor 5000"

filter rspamd proc-exec "/usr/local/libexec/opensmtpd/opensmtpd-filter-rspamd"

listen on 0.0.0.0 tls pki mail.example.com \
    filter { check_dyndns, check_rdns, check_fcrdns, senderscore, rspamd } auth-optional <passwd>

listen on ::0 tls pki mail.example.com \
    filter { check_dyndns, check_rdns, check_fcrdns, senderscore, rspamd } auth-optional <passwd>

listen on 0.0.0.0 port submission tls-require pki mail.example.com  auth <passwd> filter rspamd mask-src

listen on ::0 port submission tls-require pki mail.example.com  auth <passwd> filter rspamd mask-src

listen on 0.0.0.0 port 465 smtps pki mail.example.com  auth <passwd> filter rspamd mask-src

listen on ::0 port 465 smtps pki mail.example.com  auth <passwd> filter rspamd mask-src

action "local_mail" lmtp "/var/run/dovecot/lmtp" rcpt-to virtual <virtuals>
action "outbound" relay helo mail.example.com

match from any for domain example.com action "local_mail"
match for local action "local_mail"

match from any auth for any action "outbound"
match for any action "outbound"

The beauty of OpenSMTPD lies in its simplicity. Essentially, that's all there is to it, but here's an explanation of what this file does:

table passwd and table virtuals define tables for user credentials and virtual domains/users, crucial for authentication and email forwarding/aliasing.
pki lines specify the SSL/TLS certificate and key for encrypted connections.
filter directives apply various checks and filters to incoming connections, including dynamic DNS and reverse DNS validations, sender reputation scoring, and Rspamd filtering.
listen lines configure the server to listen on all IPv4 and IPv6 addresses for incoming SMTP and submission (port 587) connections, applying TLS and authentication as specified.
action lines define actions for email delivery, using LMTP for local delivery based on virtual mappings and setting up relay actions for outbound emails.
match rules determine how emails are processed, either delivered locally or relayed externally based on source, destination, and authentication.

To test the newly inserted configuration, run smtpd -n.

The next step is to configure Rspamd. Rather than modifying the base configuration files, we'll proceed by creating "overrides", placing specific configurations in the directory created earlier (/usr/local/etc/rspamd/local.d):

For Redis configuration in /usr/local/etc/rspamd/local.d/redis.conf:

servers = "192.168.123.4";
password = "cafecafe";

For SPF settings in /usr/local/etc/rspamd/local.d/spf.conf:

spf_cache_size = 1k;
spf_cache_expire = 1d;
max_dns_nesting = 10;
max_dns_requests = 30;
min_cache_ttl = 5m;

Now, configure DKIM signing, which rspamd will handle for every sent email. The selector must match the one entered in the DNS record previously:

In /usr/local/etc/rspamd/local.d/dkim_signing.conf:

domain {
  example.com {
    path = "/usr/local/etc/mail/dkim/example.com.key";
    selector = "mail";
  }
}

To enable support for some "known" phisher lists, create /usr/local/etc/rspamd/local.d/phishing.conf:

# Configuration options can be added here`
openphish_enabled = true;
phishtank_enabled = true;

For SURBL (Spam URI Real-time Blocklists) configuration in /usr/local/etc/rspamd/local.d/surbl.conf:

# follow redirects when checking URLs in emails for spaminess
redirector_hosts_map = "/usr/local/etc/rspamd/redirectors.inc";

For URL reputation checks in /usr/local/etc/rspamd/local.d/url_reputation.conf:

# check URLs within messages for spaminess
enabled = true;

And for caching some URL tags in Redis, in /usr/local/etc/rspamd/local.d/url_tags.conf:

# cache some URL tags in redis
enabled = true;

Rspamd should be ready for initial testing.

Now it's time to configure Dovecot, which will handle the final phase of delivery and provide mail access, as well as use Sieve for filtering and rule application.

The first step is to start with a default configuration:

cd /usr/local/etc/dovecot/ && cp -R example-config/* .

Next, modify some files. Start with conf.d/15-mailboxes.conf - add auto = create to the main folders, like this:

namespace inbox {
  # These mailboxes are widely used and could perhaps be created automatically:
  separator = .

  mailbox Drafts {
    special_use = \Drafts
    auto = create
  }
  mailbox Junk {
    special_use = \Junk
    auto = create
  }
  mailbox Trash {
    special_use = \Trash
    auto = create
  }

  # For \Sent mailboxes there are two widely used names. We'll mark both of
  # them as \Sent. User typically deletes one of them if duplicates are created.
  mailbox Sent {
    special_use = \Sent
    auto = create
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
}

Modify /usr/local/etc/dovecot/dovecot.conf to include:

protocols = imap lmtp sieve

In conf.d/10-auth.conf, switch to system authentication with passwdfile:

#!include auth-system.conf.ext
!include auth-passwdfile.conf.ext

Then, set up the passdb and userdb configuration in conf.d/auth-passwdfile.conf.ext. Replace the entire file with these contents:

passdb {
  driver = passwd-file
  args = scheme=CRYPT /usr/local/etc/mail/passwd
}

userdb {
  default_fields = uid=vmail gid=vmail home=/var/vmail/%d/%n
  args = /usr/local/etc/mail/passwd
  driver = passwd-file
}

Edit conf.d/10-ssl.conf to update ssl_key and ssl_cert:

ssl_cert = </usr/local/etc/letsencrypt/live/example.com/fullchain.pem
ssl_key = </usr/local/etc/letsencrypt/live/example.com/privkey.pem

In conf.d/20-imap.conf, include:

(...)
mail_plugins = $mail_plugins imap_sieve zlib
mail_max_userip_connections = 100

And in conf.d/20-lmtp.conf:

protocol lmtp {
  mail_fsync = optimized
  mail_plugins = $mail_plugins sieve
}

Set the mail location in conf.d/10-mail.conf:

mail_location = maildir:/var/vmail/%d/%n

maildir_stat_dirs=yes

Now, let's configure Sieve in conf.d/90-plugin.conf:

plugin {
  #setting_name = value
  sieve_plugins = sieve_imapsieve sieve_extprograms
  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox1_causes = COPY APPEND
  imapsieve_mailbox1_before = file:/usr/local/lib/dovecot/sieve/report-spam.sieve

  imapsieve_mailbox2_name = *
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve

  imapsieve_mailbox3_name = Inbox
  imapsieve_mailbox3_causes = APPEND
  imapsieve_mailbox3_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve
  sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve

  #Move the spam to Junk folder
  sieve_after = /usr/local/lib/dovecot/sieve/spam-to-folder.sieve
}

And in conf.d/20-managesieve.conf.ext:

service managesieve-login {
  inet_listener sieve {
    port = 4190
  }

  service_count = 1
  vsz_limit = 64M
}

service managesieve {
  process_limit = 1024
}

protocol sieve {
  mail_max_userip_connections = 10

  # Configuration for Sieve script management via ManageSieve protocol
}

plugin {
  sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve
  sieve = file:/var/vmail/%d/%n/sieve;active=/var/vmail/%d/%n/.dovecot.sieve
  sieve_user_log = file:/var/vmail/%d/%n/sieve/sieve_error.log
}

This setup outlines the Dovecot configuration necessary for handling mail delivery, access, and Sieve-based filtering. With these settings, Dovecot is prepared to manage both incoming and outgoing mails securely and efficiently, including support for managing Sieve scripts via the ManageSieve protocol.

Sieve scripts train Rspamd on spam and ham. Moving email into and out of the junk folder triggers an event to train Rspamd.

These files are located at /usr/local/lib/dovecot/sieve.

Create the report-ham.sieve file:

require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

if environment :matches "imap.mailbox" "*" {
  set "mailbox" "${1}";
}

if string "${mailbox}" "Trash" {
  stop;
}

if environment :matches "imap.user" "*" {
  set "username" "${1}";
}

pipe :copy "sa-learn-ham.sh" [ "${username}" ];

Create the report-spam.sieve file:

require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

if environment :matches "imap.user" "*" {
  set "username" "${1}";
}

pipe :copy "sa-learn-spam.sh" [ "${username}" ];

Create the spam-to-folder.sieve file:

require ["fileinto","mailbox"];

if header :contains "X-Spam" "Yes" {
 fileinto :create "Junk";
 stop;
}

Compile the files:

sievec report-ham.sieve
sievec report-spam.sieve
sievec spam-to-folder.sieve

Create the following two shell scripts in /usr/local/lib/dovecot/sieve.

Add the following to sa-learn-ham.sh:

#!/bin/sh
exec /usr/local/bin/rspamc -d "${1}" learn_ham

Add the following to sa-learn-spam.sh:

#!/bin/sh
exec /usr/local/bin/rspamc -d "${1}" learn_spam

Make the files executable:

chmod 755 sa-learn-ham.sh
chmod 755 sa-learn-spam.sh

Everything has been configured correctly. However, a fundamental part is missing: the users.

To begin, encrypt the password by executing:

smtpctl encrypt

Input the plain password and hit ENTER. Copy the displayed encrypted password for the next step. Now, insert the entry into /usr/local/etc/mail/passwd:

foo@example.com:your encrypted password goes here::::::

Remember to replace the placeholder with your encrypted password and ensure there are six colons at the end.

Finally, add the virtual user and associate it with the vmail system user in /usr/local/etc/mail/virtuals:

foo@example.com vmail

Note: Additional aliases can be included, such as:

postmaster@example.com foo@example.com

Let's enable and start the services:

service rspamd enable
service rspamd start
service smtpd enable
service smtpd start
service dovecot enable
service dovecot start

At this point, take a look at the logs (especially /var/log/maillog) to get an idea of what's happening. If everything is running correctly, you can now proceed with some tests. Any mail client (such as Thunderbird, etc.) should be able to connect via IMAP and send emails using SMTP, thanks to authentication. Remember that the username corresponds to the one entered in the /usr/local/etc/mail/passwd file and typically matches the full email address.

Once everything is in order, you can set the MX record of the domain "example.com" to "mail.example.com".

One of the key tests to perform is to send an email to a Gmail inbox. If the email is correctly delivered and not marked as Spam, there's a good chance that the setup is correct. There are also several online tools available that can provide useful insights into any configuration errors.

You have just installed and operationalized a complete mail server.

It is now possible to proceed with part 2, which involves the installation of Nextcloud and Snappymail, in order to have a comprehensive groupware system, while keeping one's data on their own servers.

Links:

This article was inspired by several other articles available online that provided me with a lot of useful information. Here is a non-exhaustive list: