IT Notes - openbsd

Static Web Hosting on the Intel N150: FreeBSD, SmartOS, NetBSD, OpenBSD and Linux Compared

Stefano Marinelli — Wed, 19 Nov 2025 09:16:00 +0100

Update: This post has been updated to include Docker benchmarks and a comparison of container overhead versus FreeBSD Jails and illumos Zones.

Note: Some operating systems (FreeBSD and Linux) support kernel TLS (kTLS) and the related SSL_sendfile path in nginx, which can improve HTTPS performance for static files. Since this feature is not available on all the systems included in the comparison (for example NetBSD, OpenBSD and illumos), the benchmarks were run with a common baseline configuration that does not rely on kTLS. The goal is to compare the systems under similar conditions rather than to measure OS specific optimizations.

I often get very specific infrastructure requests from clients. Most of the time it is some form of hosting. My job is usually to suggest and implement the setup that fits their goals, skills and long term plans.

If there are competent technicians on the other side, and they are willing to learn or already comfortable with Unix style systems, my first choices are usually one of the BSDs or an illumos distribution. If they need a control panel, or they already have a lot of experience with a particular stack that will clearly help them, I will happily use Linux and it usually delivers solid, reliable results.

Every now and then someone asks the question I like the least:

“But how does it perform compared to X or Y?”

I have never been a big fan of benchmarks. At best they capture a very specific workload on a very specific setup. They are almost never a perfect reflection of what will happen in the real world.

For example, I discovered that idle bhyve VMs seem to use fewer resources when the host is illumos than when the host is FreeBSD. It looks strange at first sight, but the illumos people are clearly working very hard on this, and the result is a very capable and efficient platform.

Despite my skepticism, from time to time I enjoy running some comparative tests. I already did it with Proxmox KVM versus FreeBSD bhyve, and I also compared Jails, Zones, bhyve and KVM on the same Intel N150 box. That led to the FreeBSD vs SmartOS article where I focused on CPU and memory performance on this small mini PC.

This time I wanted to do something simpler, but also closer to what I see every day: static web hosting.

Instead of synthetic CPU or I/O tests, I wanted to measure how different operating systems behave when they serve a small static site with nginx, both over HTTP and HTTPS.

This is not meant to be a super rigorous benchmark. I used the default nginx packages, almost default configuration, and did not tune any OS specific kernel settings. In my experience, careful tuning of kernel and network parameters can easily move numbers by several tens of percentage points. The problem is that very few people actually spend time chasing such optimizations. Much more often, once a limit is reached, someone yells “we need mooooar powaaaar” while the real fix would be to tune the existing stack a bit.

So the question I want to answer here is more modest and more practical:

With default nginx and a small static site, how much does the choice of host OS really matter on this Intel N150 mini PC?

Spoiler: less than people think, at least for plain HTTP. Things get more interesting once TLS enters the picture.

Disclaimer
These benchmarks are a snapshot of my specific hardware, network and configuration. They are useful to compare relative behavior on this setup. They are not a universal ranking of operating systems. Different CPUs, NICs, crypto extensions, kernel versions or nginx builds can completely change the picture.

Test setup

The hardware is the same Intel N150 mini PC I used in my previous tests: a small, low power box that still has enough cores to be interesting for lab and small production workloads.

On it, I installed several operating systems and environments, always on the bare metal, not nested inside each other. On each OS I installed nginx from the official packages.

Software under test

On the host:

SmartOS, with:
- a Debian 12 LX zone
- an Alpine Linux 3.22 LX zone
- a native SmartOS zone

FreeBSD 14.3-RELEASE:
- nginx running inside a native jail

OpenBSD 7.8:
- nginx on the host

NetBSD 10.1:
- nginx on the host

Debian 13.2:
- nginx on the host

Alpine Linux 3.22:
- nginx on the host
- Docker: Debian 13 container running on the Alpine host (ports mapped)

I also tried to include DragonFlyBSD, but the NIC in this box is not supported. Using a different NIC just for one OS would have made the comparison meaningless, so I excluded it.

nginx configuration

In all environments:

nginx was installed from the system packages
worker_processes was set to auto
the web root contained the same static content

The important part is that I used exactly the same nginx.conf file for all operating systems and all combinations in this article. I copied the same configuration file verbatim to every host, jail and zone. The only changes were the IP address and file paths where needed, for example for the TLS certificate and key.

The static content was a default build of the example site generated by BSSG, my Bash static site generator. The web root was the same logical structure on every OS and container type.

There is no OS specific tuning in the configuration and no kernel level tweaks. This is very close to a “package install plus minimal config” situation.

TLS configuration

For HTTPS I used a very simple configuration, identical on every host.

Self signed certificate created with:

openssl req -x509 -newkey rsa:4096 -nodes -keyout server.key -out server.crt -days 365 -subj "/CN=localhost"

Example nginx server block for HTTPS (simplified):

server {  
listen 443 ssl http2;  
listen [::]:443 ssl http2;  

server_name _;  

ssl_certificate /etc/nginx/ssl/server.crt;  
ssl_certificate_key /etc/nginx/ssl/server.key;  

root /var/www/html;  
index index.html index.htm;  

location / {  
try_files $uri $uri/ =404;  
}  
}

The HTTP virtual host is also the same everywhere, with the root pointing to the BSSG example site.

Load generator

The tests were run from my workstation on the same LAN:

client host: a mini PC machine connected at 2.5 Gbit/s
switch: 2.5 Gbit/s
test tool: wrk

For each target host I ran:

wrk -t4 -c50 -d10s http://IP
wrk -t4 -c10 -d10s http://IP
wrk -t4 -c50 -d10s https://IP
wrk -t4 -c10 -d10s https://IP

Each scenario was executed multiple times to reduce noise; the numbers below are medians (or very close to them) from the runs.

The contenders

To keep things readable, I will refer to each setup as follows:

SmartOS Debian LX → SmartOS host, Debian 12 LX zone
SmartOS Alpine LX → SmartOS host, Alpine 3.22 LX zone
SmartOS Native → SmartOS host, native zone
FreeBSD Jail → FreeBSD 14.3-RELEASE, nginx in a jail
OpenBSD Host → OpenBSD 7.8, nginx on the host
NetBSD Host → NetBSD 10.1, nginx on the host
Debian Host → Debian 13.2, nginx on the host
Alpine Host → Alpine 3.22, nginx on the host
Docker Container → Alpine host, Debian 13 Docker container

Everything uses the same nginx configuration file and the same static site.

Static HTTP results

Let us start with plain HTTP, since this removes TLS from the picture and focuses on the kernel, network stack and nginx itself.

HTTP, 4 threads, 50 concurrent connections

Approximate median wrk results:

Environment	HTTP 50 connections
SmartOS Debian LX	~46.2 k
SmartOS Alpine LX	~49.2 k
SmartOS Native	~63.7 k
FreeBSD Jail	~63.9 k
OpenBSD Host	~64.1 k
NetBSD Host	~64.0 k
Debian Host	~63.8 k
Alpine Host	~63.9 k
Docker Container	~63.7 k

Two things stand out:

All the native or jail/container setups on the hosts that are not LX zones cluster around 63 to 64k requests per second.
The two SmartOS LX zones sit slightly lower, in the 46 to 49k range, which is still very respectable for this hardware.

In other words, as long as you are on the host or in something very close to it (FreeBSD jail, SmartOS native zone, NetBSD, OpenBSD, Linux on bare metal), static HTTP on nginx will happily max out around 64k requests per second with this small Intel N150 CPU.

The Debian and Alpine LX zones on SmartOS are a bit slower, but not dramatically so. They still deliver close to 50k requests per second and, in a real world scenario, you would probably saturate the network or the client long before hitting those numbers.

HTTP, 4 threads, 10 concurrent connections

With fewer concurrent connections, absolute throughput drops, but the relative picture is similar:

SmartOS Native around 44k
NetBSD and Alpine Host around 34 to 35k
FreeBSD, Debian, OpenBSD around 31 to 33k
The Docker Container sits slightly lower at ~30.2k req/s, showing a small overhead from the networking layer
The SmartOS LX zones sit slightly below, around 35 to 37k req/s

The important conclusion is simple:

For plain HTTP static hosting, once nginx is installed and correctly configured, the choice between these operating systems makes very little difference on this hardware. Zones and jails add negligible overhead, LX zones add a small one.

If you are only serving static content over HTTP, your choice of OS should be driven by other factors: ecosystem, tooling, update strategy, your own expertise and preference.

Static HTTPS results

TLS is where things start to diverge more clearly and where CPU utilization becomes interesting.

HTTPS, 4 threads, 50 concurrent connections

Approximate medians:

Environment	HTTPS 50 connections	CPU notes at 50 HTTPS connections
SmartOS Debian LX	~51.4 k	CPU saturated
SmartOS Alpine LX	~40.4 k	CPU saturated
SmartOS Native	~52.8 k	CPU saturated
FreeBSD Jail	~62.9 k	around 60% CPU idle
OpenBSD Host	~39.7 k	CPU saturated
NetBSD Host	~40.4 k	CPU saturated
Debian Host	~62.8 k	about 20% CPU idle
Alpine Host	~62.4 k	small idle headroom, around 7% idle
Docker Container	~62.7 k	CPU saturated

These numbers tell a more nuanced story.

FreeBSD, Debian and Alpine on bare metal form a “fast TLS” group.
All three sit around 62 to 63k requests per second with 50 concurrent HTTPS connections.
FreeBSD does this while using significantly less CPU.
During the HTTPS tests with 50 connections, the FreeBSD host still had around 60% CPU idle. It is the platform that handled TLS load most comfortably in terms of CPU headroom.
Debian and Alpine are close in throughput, but push the CPU harder.
Debian still had some idle time left, Alpine even less. In practice, all three are excellent here, but FreeBSD gives you more room before you hit the wall.
SmartOS, NetBSD and OpenBSD form a “good but heavier” TLS group.
Their HTTPS throughput is in the 40 to 52k req/s range and they reach full CPU usage at 50 concurrent connections. OpenBSD and NetBSD stabilize around 39 to 40k req/s. SmartOS native and the Debian LX zone manage slightly better (around 51 to 53k) but still with the CPU pegged.

HTTPS, 4 threads, 10 concurrent connections

With lower concurrency:

FreeBSD, Debian and Alpine still sit in roughly the 29 to 31k req/s range
SmartOS Native and LX zones are in the mid to high 30k range
The Docker Container drops slightly to ~27.8k req/s
NetBSD and OpenBSD sit around 26 to 27k req/s

The relative pattern is the same: for this TLS workload, FreeBSD and modern Linux distributions on bare metal appear to make better use of the cryptographic capabilities of the CPU, delivering higher throughput or more headroom or both.

What TLS seems to highlight

The HTTPS tests point to something that is not about nginx itself, but about the TLS stack and how well it can exploit the hardware.

On this Intel N150, my feeling is:

FreeBSD, with the userland and crypto stack I am running, is very efficient at TLS here. It delivers the highest throughput while keeping plenty of CPU in reserve.
Debian and Alpine, with their recent kernels and libraries, are also strong performers, close to FreeBSD in throughput, but with less idle CPU.
NetBSD, OpenBSD and SmartOS (native and LX) are still perfectly capable of serving a lot of HTTPS traffic, but they have to work harder to keep up and they hit 100% CPU much earlier.

This matches what I see in day to day operations: TLS performance is often less about “nginx vs something else” and more about the combination of:

the TLS library version and configuration
how well the OS uses the CPU crypto instructions
kernel level details in the network and crypto paths

I suspect the differences here are mostly due to how each system combines its TLS stack (OpenSSL, LibreSSL and friends), its kernel and its hardware acceleration support. It would take a deeper dive into profiling and configuration knobs to attribute the gaps precisely.

In any case, on this specific mini PC, if I had to pick a platform to handle a large amount of HTTPS static traffic, FreeBSD, Debian and Alpine would be my first candidates, in that order.

Zones, jails, containers and Docker: overhead in practice

Another interesting part of the story is the overhead introduced by different isolation technologies.

From these tests and the previous virtualization article on the same N150 machine, the picture is consistent:

FreeBSD jails behave almost like bare metal and are significantly more efficient than Docker.
For both HTTP and HTTPS, running nginx in a jail on FreeBSD 14.3-RELEASE produces numbers practically identical to native hosts.
The contrast with Docker is striking: while the Docker container required 100% CPU to reach peak for the HTTP and HTTPS throughput, the FreeBSD jail delivered the same speed with ~60% of the CPU sitting idle. In terms of performance cost per request, Jails are drastically cheaper.
SmartOS native zones are also very close to the metal.
Static HTTP performance reaches the same 64k req/s region and HTTPS is only slightly behind the "fast TLS" group, although with higher CPU usage.
SmartOS LX zones introduce a noticeable but modest overhead.
Both Debian and Alpine LX zones on SmartOS perform slightly worse than the native zone or FreeBSD jails. For static HTTP they are still very fast. For HTTPS the Debian LX zone remains competitive but costs more CPU, while the Alpine LX zone is slower.
Docker on Linux performs efficiently but eats the margins. I ran an additional test using a Debian 13 Docker container running on the Alpine Linux host. At peak load (50 connections), the throughput was impressive and virtually identical to bare metal: ~63.7k req/s for HTTP and ~62.7k req/s for HTTPS. However, there is a clear cost. First, while the bare metal host maintained a small CPU buffer (~7% idle) during the HTTPS test, Docker saturated the CPU to 100%. Second, at lower concurrency (10 connections), the overhead became visible. The Docker container scored ~30.2k req/s for HTTP and ~27.8k req/s for HTTPS, slightly trailing the ~31-34k and ~29-31k range of the bare metal counterparts. The abstraction layers (NAT, bridging, namespaces) are extremely efficient, but they are not completely free.

This leads to a clear conclusion on efficiency: FreeBSD Jails provide the highest throughput with the lowest CPU cost. LX zones and Docker containers can match the speed (or come close), but they burn significantly more CPU cycles to do so.

What this means for real workloads

It is easy to get lost in tables and percentages, so let us go back to the initial question.

A client wants static hosting.
Does the choice between FreeBSD, SmartOS, NetBSD or Linux matter in terms of performance?

For plain HTTP on this hardware, with nginx and the same configuration:

Not really.
All the native hosts and FreeBSD jails deliver roughly the same maximum throughput, in the 63 to 64k req/s range. SmartOS LX zones are slightly slower but still strong.

For HTTPS:

Yes, it starts to matter a bit more.
FreeBSD stands out for how relaxed the CPU is under high TLS load.
Debian and Alpine are very close in throughput, with more CPU used but still with some headroom.
SmartOS, NetBSD and OpenBSD can still push a lot of HTTPS traffic, but they reach 100% CPU earlier and stabilize at lower request rates.

Does this mean you should always choose FreeBSD or Debian or Alpine for static HTTPS hosting?

Not necessarily.

In real deployments, the bottleneck is rarely the TLS performance of a single node serving a small static site. Network throughput, storage, logging, reverse proxies, CDNs and application layers all play a role.

However, knowing that FreeBSD and current Linux distributions can squeeze more out of a small CPU under TLS is useful when you are:

sizing hardware for small VPS nodes that must serve many HTTPS requests
planning to consolidate multiple services on a low power box
deciding whether you can afford to keep some CPU aside for other tasks (cache, background jobs, monitoring, and so on)

As always, the right answer depends on the complete picture: your skills, your tooling, your backups, your monitoring, the rest of your stack, and your tolerance for troubleshooting when things go sideways.

Final thoughts

From these small tests, my main takeaways are:

Static HTTP is basically solved on all these platforms.
On a modest Intel N150, every system tested can push around 64k static HTTP requests per second with nginx set to almost default settings. For many use cases, that is already more than enough.
TLS performance is where the OS and crypto stack start to matter.
FreeBSD, Debian and Alpine squeeze more HTTPS requests out of the N150, and FreeBSD in particular does it with a surprising amount of idle CPU left. NetBSD, OpenBSD and SmartOS need more CPU to reach similar speeds and stabilize at lower throughput once the CPU is saturated.
Jails and native zones are essentially free, LX zones cost a bit more.
FreeBSD jails and SmartOS native zones show very little overhead for this workload. SmartOS LX zones are still perfectly usable, but if you are chasing every last request per second you will see the cost of the translation layer.
Benchmarks are only part of the story.
If your team knows OpenBSD inside out and has tooling, scripts and workflows built around it, you might happily accept using more CPU on TLS in exchange for security features, simplicity and familiarity. The same goes for NetBSD or SmartOS in environments where their specific strengths shine.

I will not choose an operating system for a client just because a benchmark looks nicer. These numbers are one of the many inputs I consider. What matters most is always the combination of reliability, security, maintainability and the human beings who will have to operate the
system at three in the morning when something goes wrong.

Still, it is nice to know that if you put a tiny Intel N150 in front of a static site and you pick FreeBSD or a modern Linux distribution for HTTPS, you are giving that little CPU a fair chance to shine.

Launching BSSG - My Journey from Dynamic CMS to Bash Static Site Generator

Stefano Marinelli — Mon, 07 Apr 2025 08:11:36 +0200

I've had my own website practically forever. Back in the late '90s, I already had a web page on my ISP's server, and since at least 2001, I've had my own homepage on my own server. I've never been a great graphic designer, let alone a skilled webmaster, so I've always tried to keep things minimal and compatible.

Initially, like many others, I wrote HTML pages by hand. Then I used WYSIWYG creation tools, and eventually, I landed on CMS (Content Management Systems).

The Era of Dynamic CMS

I liked CMS because they allowed me to focus on the content and not on the correctness of the generated HTML. Thanks to them, I started writing my first blog shortly afterward.

Over the years, I've used many tools like PHPNuke, FlatNuke (created and developed by my friend Simone Vellei), eventually moving through Joomla and Wordpress. Wordpress always seemed like the most suitable tool for the job, and I used it for many years. Even today, mainly on the sysadmin side, I manage hundreds of Wordpress sites, and they are reasonably reliable, aside from the plugins (because the problem with Wordpress isn't the software itself, but many of the external plugins).

But this is precisely the problem: all dynamic CMS require constant and continuous security updates because, without them, the chances of defacement are extremely high.

Discovering Static Site Generators

And that's precisely why, when I discovered Carlos Fenollosa's bashblog in 2014, it immediately became clear that, indeed, there was no reason to continue down the path of dynamic CMS. I don't write often, I don't update often, there's no reason to regenerate all the content with every visit. Sure, WordPress caching plugins are often quite effective, but they are still add-ons that need to be kept up to date. And I'm not a fan of adding things to streamline. Often, less is more.

So, I started using bashblog for some 'secondary' projects until, in 2015, I migrated my 'old' Italian blog from WordPress to Pelican. Shortly after, I moved from Pelican to Nikola, and that blog is still generated by Nikola, although (that blog's) updates are now extremely rare (so much so that I consider it almost abandoned). I also created the first Docker container for Nikola and, for a long time, it was listed among the deployment methods on their site.

Building My Own: BSSG

But bashblog continued to fascinate me. So in 2015, for fun, I started developing my own Static Site Generator from scratch. I called it (with little imagination), BSSG - Bash Static Site Generator. The plan was for it to be compatible with the main OSes I use, to remain sufficiently simple and straightforward (!!!), and to be tailored to my needs. I intended to use it only and exclusively for small private things, starting with a sort of diary of mine - more professional than personal - and leave the 'official' blogs to more tested and 'professional' tools.

As time went by, I added some small features I liked: theming support, archives, tags (initially absent). Over time, many functions were added, and the script grew large – large enough to make me pause and ask myself some questions about the long-term stability of this solution. So, it remained only for my 'diary', which, however, grew year after year to the point where I needed to devise some kind of optimization. I then developed (more for fun than out of real necessity) a caching system. On rebuild, only what needs to be rebuilt is reconstructed, making the operation sufficiently fast even as the number of posts grows. Obviously, there are limits: using bash and external tools, the efficiency cannot be compared to that of a proper programming language.

Brief Detour: ITNBlog

And it's here that I decided, in preparation for opening a new blog (this one), to create a new tool called ITNBlog. I would develop it in Python and focus a bit more on performance and completeness. But ITNBlog stalled very quickly: time was limited, I'm not a full-time developer, so I realized I would spend too much time on development and too little on content creation.

Therefore, in 2018, I launched this blog but using Ghost, a solution that gave me good results, including performance-wise. I chose Ghost because I thought that, writing content also from my phone while on the go, a real CMS would be useful. Spoiler: no, it didn't turn out that way, so a few years later I decided to migrate this blog to Hugo. Nevertheless, I continued to develop ITNBlog on and off, as a hobby, without any particular ambitions.

At some point, however, I found myself in a particular situation: Hugo deprecated some features, and the theme I had chosen moved forward. But I ended up in an unpleasant situation: using the latest version of Hugo and the current version of the theme would produce unacceptable output; staying with the old version of Hugo while waiting for the theme update meant making a compromise. I actually build the blog from different devices, and they all have different versions of Hugo installed. Change the theme? Feasible, but I would have had to modify almost the entire site.

I considered migrating to manpageblog by gyptazy – I personally love its simplicity and retro look, and it was the main candidate to replace Hugo. I also created a script and migrated all my posts into the correct format.

BSSG to the Rescue (and ITNBlog's Role)

That's when I realized: I would implement the few missing features needed to make ITNBlog sufficiently complete, and this blog would be published using it, ensuring I'd be committed to its development. However, ITNBlog is not mature enough to be released publicly, so for now, it will remain the engine just for my blog. Then I thought again about BSSG – development had stalled some time ago, but it was still in use – and figured that perhaps, with a little tidying up, I could release it.

Because I'm tired of seeing people use dynamic CMS even to implement primarily static blogs or websites – and BSSG, despite its limitations and inefficiencies, works. And there are many themes to choose from. In short, you can install it and generate your blog in seconds.

Why Choose BSSG?

BSSG is the result of a 10-year evolution. The code isn't extremely consistent, some interesting features are missing (which I plan to implement), and it could use refactoring as the build script is monstrously large. But it works, it's portable (and much of the complexity increased precisely because of portability), and it generates sites that achieve very high accessibility and speed scores.

Here are some highlights:

✅ Portability: Uses native OS tools (e.g., md5sum on Linux, md5 on OpenBSD and NetBSD). Portability itself added much of the complexity!
✅ Simple Theming: Themes are just simple CSS files, so the structure remains the same – simplifying theme switching or creating new ones. More than 50 themes are already available!
✅ Essential Features: Supports RSS feed generation, sitemap.xml, OpenGraph tags (to improve social sharing), internationalization (the blog can be in languages other than English – but not multilingual, at least for now), etc.
✅ Built-in Backup and Restore script: It will just copy the configuration file, posts, and pages. Nothing else.
✅ Minimal Dependencies.
✅ Markdown Support: Posts and pages are in Markdown (CommonMark, Pandoc, and markdown.pl are supported).
✅ Feature Images.
✅ Optional GNU Parallel Integration: To speed up build times when there are many posts. This feature significantly impacts the code and has caused me numerous headaches over time. But it's optional (if parallel isn't found, it proceeds traditionally) and only provides benefits when the number of posts increases: with few posts, performance actually degrades.
✅ High Accessibility and Performance Scores: Sites built with BSSG achieve excellent scores.
✅ BSD Licensed: Released under a BSD license.

One of the problems I've always had with all CMS and SSGs has been choosing a theme. In some cases (like Hugo), the theme heavily influences the output, which is both good and bad. Good because it makes each site unique, but bad because it makes switching themes difficult. In the past, I've sometimes found myself having to change themes because they were abandoned and no longer updated. BSSG works differently: theming comes from using a different CSS file, which makes its structure more rigid, but switching from one theme to another is trivial. To help with the choice, I created a script that will build your site using all the themes present in the themes directory, just like on the examples page of the official website. This way, it will be easy to see and test your site with all available themes. If you want to add a touch of originality, you can choose the 'random' theme, and one will be chosen randomly from the list at each site regeneration.

Admin Interface (Experimental)

BSSG is in production use by some clients (for their internal sites), for whom I also created a basic admin interface (using Node Express, partly to chew on a bit of Node), but I don't feel ready to release it immediately as it's not sufficiently tested. It has an integrated Markdown editor and allows post scheduling, generating the files and launching BSSG with the right options at the right time. This could be that connecting link between traditional CMS and SSGs. There are others, but this one is tightly integrated with BSSG.

BSSG is Available Today

Starting today, BSSG is publicly available. It's not perfect, it probably doesn't make sense to do something of this complexity in bash, development will proceed slowly – but it's here, available to anyone who might find it useful.

Happy blogging everyone!

OSDay 2025 - Why Choose to Use the BSDs in 2025

Stefano Marinelli — Sun, 23 Mar 2025 10:30:00 +0100

This is the text underlying my presentation at OSDay 2025, held on 21 March 2025 in Florence, Italy. There was limited time, so I couldn't go into much detail and had to keep things more general and structured than usual. You can watch the video of my talk on YouTube.

The slides can be downloaded here

Happy reading!

OSDay Florence - 21 March 2025 - Why Choose to Use the BSDs in 2025

"I'm Stefano Marinelli, I solve problems."

I'm the founder and Barista of the BSD Cafe, a community of *BSD enthusiasts.

I work in my company, called Prodottoinrete - a container of ideas and solutions.

I'm passionate about technology and computing, and I've made my passion my profession. Every morning, when I sit in front of the computer, a new world opens up for me to explore.

I've been a Linux user since 1996, before I turned 17. Back then, I used Fidonet and would read about alternative operating systems. I experimented with Linux distributions from CDs, and by 1997, Linux became my everyday system. It was only in 2002 that I began exploring BSD systems, largely thanks to FreeBSD's fantastic handbook.

The relationship we had with Open Source 20-30 years ago was fundamentally different than today. Back then, embracing Open Source meant thinking differently. It meant embracing freedom. We chose Linux and the BSDs when Windows and commercial Unix systems dominated the market. Not because they were simple or free (as in free beer), but for freedom from impositions - both technological and ideological.

I solve problems. And to solve problems effectively, we need to recognize when the landscape has changed.

The reality today is that while we won that war - Open Source is everywhere - we're facing a new challenge. The "mainstream" Open Source world is creating monocultures. The focus has shifted from technologies to specific tools. We're seeing innovation for novelty's sake, not problem-solving.

This shift has profound implications. In a world dominated by cyber threats, where everything is connected and we completely depend on technology, the value of stability has been lost. By stability, I don't just mean that a system doesn't crash. I mean continuity over time, upgradeability, and system visibility.

Instead, the industry seems obsessed with the hype cycle. "New" is prioritized over secure and stable. The mantra has become: - "It will be fixed in the next version" - "We need automatic restarts when it crashes" - "Do we need software that crashes less? We have systemd and Kubernetes to restart crashed workloads!" - "We need moooarrr powaaaaaaar!!!!"

Let me give you a concrete example. A program written in Rust should be memory safe - that's one of the main selling points of the language. But if that program uses unsafe functions and segfaults, what advantage does it offer over a mature C implementation? Stability matters more than the implementation language.

I solve problems. And creating a monoculture does not solve problems - it creates new ones.

Yes, Linux, Docker, and Kubernetes are better than closed source solutions. But when everyone uses the same tools, freedom dies. We use them because "everyone does" rather than because they're the best tool for our specific needs.

If we had only used what everyone else used, we wouldn't have Linux or the BSDs today. There would be no LibreOffice, no Nextcloud. We'd just have Windows variations and expensive Unix systems. We'd be bound by licenses and vendors, stuck with closed solutions.

This is where the BSDs offer a compelling alternative: "Be free and evaluate alternatives. Always."

For those who don't know, the original BSD started in the 1970s (before Linux was conceived). Minix was created as an educational OS because it was believed that BSD, mature and professional, would be the Open Source OS that would dominate the market. A legal case stalled development and scared adopters, but in 1993, NetBSD and FreeBSD emerged. OpenBSD forked from NetBSD later, then DragonflyBSD from FreeBSD.

As Linus Torvalds said in 1993, "If 386BSD had been available when I started on Linux, Linux would probably never had happened."

What makes the BSDs special is their philosophy: - Kernel and userland developed by same teams - Consistency in tools and updates - Excellent documentation - especially OpenBSD, where insufficient docs are considered a bug - Man pages contain virtually everything - Evolution, not Revolution

Let me briefly introduce the main BSD variants that I work with daily:

FreeBSD is a generalist system. It focuses on stability and performance - with HardenedBSD as a security-enhanced fork. It has native ZFS, Boot Environments, and complete separation between OS and packages. It's had container support via jails since 2000 - which predates Linux cgroups by a decade! It offers bhyve virtualization (more efficient than KVM). OPNsense and pfSense are based on FreeBSD, as pf is a powerful firewall. It's used by Netflix for streaming video delivery and forms the foundation for PlayStation consoles. MacOS and iOS also contain some FreeBSD code.

OpenBSD focuses on security and code correctness. Its code is constantly audited and simplified - less is more. The team believes "The more complex the code, the less maintainable." It has security mechanisms like pledge() and unveil(). OpenSSH (and many other nice things) originated and are developed here. Development is driven by team priorities, not user requests. It's ideal for routers, firewalls, and security-critical systems.

NetBSD lives by the motto "Of course it runs NetBSD!" Its focus is on correctness, portability, and proper implementation. It supports 50+ architectures. Development centers on compatibility, which necessitates code quality. It must function on decades-old hardware. It's ideal for systems that require stability without the need for continuous updates, like embedded devices.

I solve problems. And in my experience, the BSDs have consistently proven to be excellent problem-solvers. Here are some real-world benefits I've experienced:

Better stability and security
Simplified administration - upgrades won't destroy your system
Less vulnerability to common attacks - "We don't need this patch, you're running OpenBSD and it's been fixed 20 years ago"
Network interfaces maintain consistent names - ix0 will remain ix0, not renaming from enx3e3300c9e14e to enp10s0f0np0
FreeBSD shows lower system load compared to Linux
FreeBSD handles I/O pressure better - on the same hardware, I've seen 70% time reduction
FreeBSD delivers improved end-user experience/responsiveness
NetBSD provides the comfort of "Don't worry - your platform will be supported for the foreseeable future"

So why choose BSD in 2025? I believe there are several compelling reasons:

Security in an increasingly hostile environment
Stability in a world obsessed with novelty
Performance without unnecessary complexity
Freedom from the mainstream monoculture
Systems designed with coherent philosophy

Don't be afraid to try BSD systems - despite the Beastie mascot, they don't hurt and you'll appreciate them!

See you at BSD Cafe!

I Solve Problems

Stefano Marinelli — Thu, 03 Oct 2024 08:53:00 +0200

Addendum: during the event, I was interviewed by Jason Tubnor for the BSD Now Podcast, where I provided further information about the talk and the BSD Cafe project. Here is the link to the episode

This is the text underlying my presentation at EuroBSDCon 2024, on 21 September 2024, in Dublin, Ireland and BSDCan 2025, 13 June 2025, Ottawa, Canada.

The slides can be downloaded here

The BSDCan 2025 video can be viewed here. This is more up-to-date than the EuroBSDCon one.

The EuroBSDCon 2024 video, not yet separated from the live stream, can be viewed here - At first, I was a bit tense, then I relaxed.

Happy reading!

EuroBSDCon Dublin - 21 September 2024 - Why (and how) we're migrating many of our servers from Linux to the BSDs

"I'm Stefano Marinelli, I solve problems."

I’m the founder and Barista of the BSD Cafe, a community of *BSD enthusiasts.

I work in my company, called Prodottoinrete - a container of ideas and solutions.

I’m passionate about technology and computing, and I’ve made my passion my profession. Every morning, when I sit in front of the computer, a new world opens up for me to explore, and I try to share this passion with my clients. Sometimes, I succeed.

I've been a Linux user since 1996, before I turned 17. Back then, I used Fidonet and would read about alternative operating systems. Curiosity got the best of me, and I bought a set of CDs with various Linux distributions at the first opportunity. I tried it, I liked it, but at the time, I didn’t find it advantageous, so I continued using it for secondary tasks while Windows remained my daily driver.

Things changed in late 1997. I decided to go deeper into Linux and, with the purchase of a new, more powerful computer, I realized that aside from gaming, Linux could be my everyday system. This came in handy when I started university in 1998, where the computer science department was very oriented towards Open Source solutions. I was one of the few students who already understood the concepts of Open Source and knew how to use Linux. I was also one of the few who wasn’t baffled when we found Solaris machines in the lab. After all, there were similarities.

Over the years, I became one of the administrators of that lab, which was almost entirely Linux-based.

In 2000, I was fortunate to have Professor Özalp Babaoğlu as my lecturer, which pushed me to explore other operating systems like the BSDs. However, all I had at the time was an old Compaq laptop (486/25 MHz and 4 MB RAM) and no fast internet connection. So, apart from some theoretical studies, I postponed hands-on experiments until I had better resources. By 2002, thanks to a broadband connection and a new computer, I began exploring BSD systems. I started with FreeBSD, largely thanks to its fantastic handbook. I asked my parents to buy a laser printer so I could "print academic material" - but really, I wanted to print all the documentation I could find, starting from the FreeBSD handbook. And it was incredibly helpful.

Before long, FreeBSD became my daily driver - entire nights spent compiling KDE while I slept a meter away from my laptop’s wildly spinning fans - but FreeBSD ran much, much better on that machine than Linux did. Unfortunately, OpenBSD was too slow to be usable with any graphical interface.

In 2003, my final thesis focused on virtualization on Open Source systems, NetBSD/Xen was one of the best and most efficient solutions I tested.

Shortly after, I found a job at a company that was just beginning to offer Linux-based solutions, mainly for web and mail servers, but I was criticized because I completed tasks that didn’t require constant interventions - this hurt the company’s billing. That’s when I set my future guidelines:

I would work for myself, following my own philosophy.
I would not be tied to any particular vendor. I love exploring and learning, so I would always study solutions in depth and recommend the one I thought best suited for the client.
I solve problems - I don’t sell boxes.
I would use and promote Open Source solutions whenever possible.
I would use a BSD whenever feasible and Linux where a BSD was not suitable.

In every technological decision, my priority is solving my clients’ specific problems, not selling a predefined solution. I solve problems.

I was often told that Open Source systems were “toys for universities” and that the real world ran on something else (mostly meaning Windows). I pressed on. In some cases, I offered to be paid only if the results were achieved, showing how much they’d save compared to traditional licensing costs. It worked, but...

They accepted Linux, albeit reluctantly, but rejected BSDs because they didn’t know them. In some cases, I managed to convince them. In others, sadly, I didn’t.

Result: I decided to use the BSDs where clients didn’t have direct access - like email servers or web hosting - while using Linux where they specifically requested it.

NetBSD/Xen as the virtualization base, with excellent longevity and reliability.
OpenBSD as network/firewall entry points.
FreeBSD (especially with the introduction of ZFS) for various services, including backup servers.

It worked. What surprised clients most was the stability and the reduced need for maintenance. They saw more uptime and were happy.

Problem: "If nothing is working, what am I paying you for? If everything’s working, what am I paying you for?" So, I selected clients and situations that understood that if everything works, it’s because of the work behind the scenes. It’s better to pay for everything to work than to pay to fix problems. The problem to solve, in this case, is not stopping the client’s work. And I solve problems.

I managed about 65% *BSD machines and 35% Linux. But as Linux’s popularity grew, so did client demand. It became necessary on multiple occasions to replace or implement Linux instead of a BSD, for specific requests.

At a certain point, Linux virtualization solutions matured, and many of my hosts transitioned from NetBSD/Xen to OpenNebula (often with MooseFS) and then to Proxmox (often with Ceph). This happened because my clients needed to manage their VM lifecycles autonomously, change configurations, migrate hosts, etc.

Proxmox showed excellent reliability and stability. The VMs were often Linux-based (especially if the client needed direct management) or FreeBSD, but in smaller quantities. My own infrastructure, however, remained primarily BSD-based.

Over time, I gradually started to reflect and 'take stock'.

No *BSD has ever caused me to lose data to the point of having to restore from backup. On Linux, I’ve lost data with ext4, XFS, and btrfs. The most catastrophic case was with XFS - just a few files, backed up and restored, but the client was highly demanding and didn’t take it well. The largest failure was with btrfs - after a reboot, a 50 TB filesystem (in mirror, for backups) simply stopped working. No more mounting possible. Data was lost, but I had further backups. The client was informed and understood the situation. Within a few days, the server was rebuilt from scratch on FreeBSD with ZFS - since then, I haven’t lost a single bit.

In 2018, I started introducing Docker and Podman to the developers with specific needs, especially to help them with their development. Many developers began incessantly asking for Docker, unfortunately some of them only wanted to bypass many limitations that traditional setups imposed. Unfortunately, in many cases, those "limitations" were just bad practices - running outdated versions of software and libraries, or not wanting to bother with keeping the stack updated. There are similar problems with other solutions, but at least I can block them. They're great for quick deployment and increased security, but they aren’t always the best choice. Plus, they’re not the only option, despite what many people think these days.

Linux has had major development over the past years, but this has shifted towards specific players’ interests (mainly cloud providers) rather than technical reasons. Maybe not in the kernel, but many Linux distributions and communities now seem focused on the constant push to replace "the old with the new" with solid theoretical reasons but sometimes seemingly without practical benefit.

For me, computing should solve problems and provide opportunities to those who use it. I solve problems. Every change or variation will solve one problem but create new ones. It’s crucial to be mindful not to create worse problems than the ones you’re solving, and today’s enterprise world often seems to overlook that.

It’s common to see software distributed solely via Docker Compose files. Sometimes I use it as an installation tutorial, but I realize they’re just specific pieces precariously glued together (patched files, specific dependency versions, or nothing works).

The trend is to rush, to simplify deployments as much as possible, sweeping structural problems under the rug. The goal is to "innovate", not necessarily improve - just as long as it’s "new" or "how everyone does it, nowadays".

A massive business has grown around Linux: certifications, training, pentesting, certified platforms - the community is losing decision-making power. This is a far cry from the early days.

I solve problems. And these fast-changing technologies risk creating more problems than they solve, at least in certain situations. The workloads I manage often stay up for years, requiring a more stable, upgradable, and consistent approach.

If a client’s problem is to have an e-commerce site, they don’t really care if it runs on Docker, Podman, a FreeBSD jail, or a Raspberry Pi cluster - as long as their problem is solved. In fact, clients are happy when their solution is stable, upgradable, and secure.

There isn’t a single solution to all problems, but many solutions for each problem. My job is to give clients the best solution to solve their specific problem, not the most fashionable one.

That’s why I decided, several years ago, to reverse the proportion and implement and the BSDs for all possible workloads.

Each *BSD has its characteristics and target audience. Sometimes these targets overlap, but it’s generally not difficult to choose the most appropriate solution.

The goal of the migration was to create stable, coherent, upgradable, and secure systems.

By implementing an OpenBSD system, I often don’t need to install any additional packages. When it’s time to upgrade, it’s simple and secure. When a vulnerability emerges, I’m often fortunate to read "OpenBSD is excluded from this issue because it eliminated this risk X years ago..."

By implementing a NetBSD system, I know I’m installing a system that isn’t in a rush to release new versions and will likely run for years with only a few package updates and security patches, when necessary. And it’s quite rare for a patch to be required.

By implementing a FreeBSD system, I know I’ll have ZFS at my disposal, a fast and efficient hypervisor like bhyve, and a native, mature jail system that will ensure service and setup separation coherently and precisely. Not as “add-ons” but built into the system itself.

Moving to the BSDs means, in my experience, reaching systems that are more stable, more easily upgradable, and more consistent in their parts. They don’t chase the hype of the moment, much like the early days of Linux. In the case of FreeBSD, this also means moving to native ZFS and boot environments, giving me greater peace of mind when it comes to upgrades.

The initial strategy was to migrate what would soon need updates anyway, what would be moved (thus requiring a new setup), and what was causing problems and deserved a deeper dive.

First, I decided to migrate to FreeBSD the hypervisors not directly accessed by clients, especially on leased servers. The approach was to create a twin machine (to have an objective performance and stability comparison - it wouldn’t make sense to compare it to a different machine), install FreeBSD, bridge it with the production server, install vm-bhyve, and start copying VMs from Proxmox, reconfiguring the main parameters in the configuration file. In some cases, I already used ZFS on Proxmox, so the first part of the migration was a simple zfs-send/receive. In other cases (e.g., when using Ceph), I made an intermediate move by live-migrating the storage to ZFS and then proceeding as in the first case. The first noticeable effect was a reduction in resources used by the host to handle VM traffic - as expected, only FreeBSD’s basic processes and bhyve were running - but at the same time, there was a significant increase in I/O performance, further enhanced by switching the virtual disk driver from virtio to NVMe. This allowed for in-depth testing and revealed that FreeBSD suffers less under heavy I/O (the VMs on Linux tended to block their I/O, an effect I didn’t notice on FreeBSD) and showed significantly lower loads. In short, it handled the load better.

As an experiment, I decided to migrate two hosts (each with about 10 VMs) of a client - where I had full control—without telling them, over a weekend. By Tuesday, they called me, concerned: they had noticed a massive performance boost and were worried I had upgraded their hardware without approval, thinking that, "given the performance boost", it would cost them a lot. After explaining what I had done, they asked me to run further tests and progressively continue migrating everything. 20 hosts, all based on (sometimes slightly older) versions of Proxmox. And so I did, but taking advantage of their open-mindedness, I went a step further.

Many of their VMs handled simple workloads - PHP websites, or Java-based management systems (running on Tomcat), device monitoring software for industrial machinery, etc. The decision to use VMs had been made by their highly competent internal IT staff to separate environments and dependencies. It was a perfect use case for jails. I decided to try, VM by VM, to replicate the setups inside FreeBSD jails. In some cases, for convenience, I managed to run everything directly in Linux jails (with Linuxulator); in others, it was impossible, so I recreated the setups in individual jails. They immediately noticed an effect: faster operations. It didn’t surprise me. We avoided double buffering (VM and OS), so all the saved RAM could be used by ZFS for its cache and by the host to run other services. Some VMs remained as VMs (e.g., Zimbra). By the end of the operation, I had drastically reduced the number of VMs, replaced by jails, and consequently, the number of hosts. From 20 down to 11 - with a significant monthly cost saving.

The road was now clear, and I progressively continued down this path. One of the most interesting anecdotes: a client told me that they used to start an operation before taking a coffee break, around 15 minutes, to find the task almost done by the time they returned. After the migration, they shared that they launched the process, grabbed their things, and the task was already complete. An estimated reduction from about 18 minutes to 6 minutes on average. I didn’t investigate too much, but I suspect a combination of factors, with the predominant one being bhyve’s NVMe driver.

The main challenge I often face is ideological. Some people are used to thinking that the ideal solution is X - and believe that X is the only solution for their problems. Often, X is the hype of the moment (a few years ago, I fought to convince people that VMware wasn’t necessary and that Proxmox would be a great solution; today, Proxmox is on everyone’s lips - but it’s not the only solution). Often, X is a "cloud" cluster with Kubernetes - running WordPress on it. Even for hosting a law firm’s website, which will be updated every five years.

When I ask, "Okay, but why? Who will manage it? Where will your data really be, and who will safeguard it?", I get blank faces. They hadn’t considered these questions. No one had even mentioned them. "But everyone I spoke to proposed this type of solution...". It’s like at the beginning with Windows, when I proposed *BSD or Linux. Or later with VMware, when I proposed Proxmox. Or now with Kubernetes, when I propose the BSDs.

But the simplest solutions are the easiest to maintain and manage over time. My experience has taught me that setting something up is often the easiest part. The hardest part is returning to it after 1, 5, 10 years. Keeping it running, updating it, stabilizing it. For many, IT isn’t their business, but a tool to achieve their goals. A Kubernetes cluster is fantastic, but it requires maintenance. Or it’s external - so it’s no longer ours. We’ve lost control of the data. For many, it’s unnecessary to complicate things. And with every additional layer, we’re creating more problems.

No BSD system has ever surprised me during an update or a simple reboot. I’ve never encountered, for example, a network interface renaming from enx3e3300c9e14e to enp10s0f0np0* on Linux, effectively locking me out of a server. ix0 will remain ix0.

I’ve never had to recompile ZFS on FreeBSD, only to find that the module wouldn’t load, blocking the filesystem from mounting after reboot.

Many of the developers I work with have embraced the challenge. Most of them are passionate about technology, and learning a new operating method has been very interesting. Almost all of them, after experiments, were positive and, in fact, began explicitly requesting "jails" instead of Docker hosts. They started using BastilleBSD to clone "template" jails and deploy them. They learned to access ZFS automatic snapshots and recover lost files. They learned to manage the resources at their disposal without repeating the usual mantra of "we need mooooar powaaaar!" every time there’s a problem, a slowdown, or a storage overload. They’ve returned to trying to understand what’s happening, rather than just assembling pieces, libraries, and containers without considering the effects.

Others, however, struggled, but remained positive nonetheless. And that’s okay. I solve problems and can’t force my solutions on everyone.

In other cases, the challenge wasn’t technical but "commercial". Often, decision-makers have little to no technical knowledge. Linux sells well. "Cloud" sells even better. A "NetBSD-based solution", unfortunately, has less commercial appeal today. So, they want what they can sell, without focusing too much on the advantages of alternative solutions.

Linux today is subject to many compliance requirements - I’m often asked which version of OpenSSH I’m running - and they complain when the version (the latest from OpenBSD) isn’t considered "secure" because it doesn’t match their procedures (e.g., OpenSSH_9.2p1 Debian-2+deb12u3). They don’t understand when I explain that the "Debian" part refers to the Linux distribution, not a release. Those who prepare these documents are often, sadly, unaware of what they’re asking for. They just have a checklist.

The transition is ongoing, and as I see opportunities, I’m migrating from Linux to the BSDs whenever possible. Today, I can say that 78% of the hypervisors I manage run on FreeBSD, and 66% of the workloads (VPS, jails, hosts, etc.) are running on one of the BSDs - including solutions like OPNsense. None of the clients have experienced major issues. No one has complained about performance or reliability. All feedback has been positive. Many appreciate moving away from IT monocultures, especially when problems arise - following the recent severe SSH vulnerability, many clients contacted me, worried. It was nice to tell some of them that the exposed SSH service was running OpenBSD’s version, and that OpenBSD wasn’t affected as they had developed a secure mechanism back in 2001. They appreciated it. They want more setups based on OpenBSD.

I'm Stefano Marinelli, I solve problems. And I love solving problems using BSD systems.

Make Your Own CDN with OpenBSD Base and Just 2 Packages

Stefano Marinelli — Thu, 29 Aug 2024 01:41:00 +0200

Introduction

This article is a "spin-off" from the previous post "Building a Self-Hosted CDN for BSD Cafe Media," which I recommend reading, based on FreeBSD. In that article, I showed how I addressed the issue of geo-replication and geo-distribution of BSD Cafe media content. If you prefer a NetBSD-based setup, there's an article that describes how to do it.

The internet today relies TOO MUCH on just a few big players. When one of them stops working, half the world is impacted because too many services, in my opinion, depend on them. “Too big to fail,” some might say. “Single Point of Failure,” I respond."

The strength of the internet has always been its extreme decentralization, which is now less evident due to this phenomenon.

In this article, I want to show how easy it is to create a self-hosted CDN using OpenBSD and just two external packages: Varnish and Lego.

Actually, only one package is truly needed (Varnish), and SSL certificates could be generated using the built-in acme-client in OpenBSD. However, this might be limiting since acme-client handles certificate generation via traditional methods (using a file in .well-known), but when dealing with a CDN and several reverse proxies listening, you don’t have perfect control over which one will receive the certificate generation validation request.

The choice of Varnish is based on several factors, with the main ones being the ability to keep the cache in RAM (which means it can run on read-only systems) and the ability to flush the cache remotely. For example, with each change to my blog, I can choose whether to perform an immediate flush (such as for a new article or an error) or wait for the cache's "natural" expiration (such as for a typo or minor, non-critical changes).

For convenience and practicality, I’ll use the excellent Lego tool—a Go application that supports many DNS authentication methods, including PowerDNS.

Installation

The steps are quite simple. After installing and updating OpenBSD (using the syspatch command), start by installing the two packages:

obcdn# pkg_add lego varnish
quirks-7.14:updatedb-0p0: ok
quirks-7.14 signed on 2024-08-24T13:35:23Z
quirks-7.14: ok
lego-4.16.1: ok
varnish-7.4.2:bzip2-1.0.8p0: ok
varnish-7.4.2:pcre2-10.37p2: ok
useradd: Warning: home directory `/var/varnish' doesn't exist, and -m was not specified
varnish-7.4.2: ok
The following new rcscripts were installed: /etc/rc.d/varnishd
See rcctl(8) for details.

The next step is to enable Varnish:

rcctl enable varnishd

Varnish has a generic startup script, but it's best to customize the startup options. To do this, modify the /etc/rc.conf.local file:

varnishd_flags="-j unix,user=_varnish,ccgroup=_varnish -f /etc/varnish/default.vcl -T localhost:9999 -a localhost:8080 -s default,500m"

This configuration sets Varnish with a 500 MB cache and listens on localhost, port 8080.

Next, rename the default VCL file to prepare for your own content:

mv /etc/varnish/default.vcl /etc/varnish/default.vcl.distrib

Create a new default.vcl file. Below is an example based on this blog (at the time of writing). You’ll need to adapt it according to your needs, especially if there are cookies or other dynamic content. Note that Varnish will fetch data from a specific backend accessed in http. If privacy is needed, consider creating a VPN between the backend and Varnish, as briefly mentioned in the previous article:

vcl 4.1;
import std;

# Backend - it-notes.dragas.net
backend it_notes {
    .host = "myOriginalServer";
    .port = "80";
}

# ACL - purge - it-notes.dragas.net
acl purge_it_notes {
    "authorizedIPForCachePurge";
}

sub vcl_recv {
    # it-notes.dragas.net
    if (req.http.Host == "it-notes.dragas.net") {
        set req.backend_hint = it_notes;
        set req.http.Host = "it-notes.dragas.net";

        # PURGE - it-notes.dragas.net
        if (req.method == "PURGE") {
            std.log("Purge request received for " + req.url);

            if (!std.ip(req.http.X-Forwarded-For, "0.0.0.0") ~ purge_it_notes) {
                return (synth(405, "Not allowed."));
            }

            if (req.url == "/" || req.url == "/*") {
                ban("req.http.host == " + req.http.host);
                return(synth(200, "Entire cache has been cleared."));
            }
            return (purge);
        }

    } else {
        # Other domains - 404
        return (synth(404, "Domain not found"));
    }

    if (req.method != "GET" && req.method != "HEAD") {
        return (pipe);
    }

    return (hash);
}

sub vcl_backend_response {
    # TTL - it-notes.dragas.net
    if (bereq.http.host == "it-notes.dragas.net") {
        if (bereq.url ~ "\.(gif|jpg|jpeg|png|webp|ico|css|js)$") {
            set beresp.ttl = 1w;
            set beresp.grace = 1d;
            set beresp.keep = 7d;
            unset beresp.http.Set-Cookie;
            unset beresp.http.Cache-Control;
            set beresp.http.Cache-Control = "public, max-age=604800";
        } else {
            set beresp.ttl = 15m;
            set beresp.grace = 48h;
            set beresp.keep = 7d;
        }
    }

    # Remove some headers
    unset beresp.http.Server;
    unset beresp.http.X-Powered-By;
    unset beresp.http.Via;

    return (deliver);
}

sub vcl_deliver {
    # Add X-Cache header
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
    } else {
        set resp.http.X-Cache = "MISS";
    }

    std.log("Delivering content for " + req.url + " - Cache: " + resp.http.X-Cache);

    # Remove Varnish headers
    unset resp.http.Via;
    unset resp.http.X-Varnish;

    return (deliver);
}

sub vcl_hash {
    hash_data(req.url);
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
    return (lookup);
}

sub vcl_hit {
    return (deliver);
}

sub vcl_miss {
    return (fetch);
}

sub vcl_purge {
    std.log("Purge executed for " + req.url);
    return (synth(200, "Purge successful"));
}

sub vcl_synth {
    set resp.http.Content-Type = "text/html; charset=utf-8";
    set resp.http.Retry-After = "5";
    synthetic({"<!DOCTYPE html>
        <html>
            <head>
                <title>"} + resp.status + " " + resp.reason + {"</title>
            </head>
            <body>
                <h1>Status "} + resp.status + " " + resp.reason + {"</h1>
                <p>"} + resp.reason + {"</p>
                <h3>Guru Meditation:</h3>
                <p>XID: "} + req.xid + {"</p>
                <hr>
                <p>Varnish cache server</p>
            </body>
        </html>
    "});
    return (deliver);
}

Start Varnish and check if it starts correctly:

rcctl start varnishd

Generating SSL Certificates

Before configuring relayd, you’ll need to generate SSL certificates. Lego supports many DNS providers and provides clear and comprehensive examples, so I suggest reading its README file.

Important: Certificates generated by Lego are not directly compatible with relayd, so you must generate them in the correct format. Add the -k rsa4096 flag to the Lego command to obtain certificates compatible with relayd.

Once generated, the certificates will be in a subdirectory of the directory from which the command was launched. For example, if the command is run as root (which is unnecessary, but just for the example), the certificates will be in /root/.lego/certificates/.

relayd expects certificates in a specific location. Copy them to the appropriate directories. In my example:

obcdn# cp /root/.lego/certificates/it-notes.dragas.net.crt /etc/ssl/
obcdn# cp /root/.lego/certificates/it-notes.dragas.net.key /etc/ssl/private/

Remember to copy the files to the correct directories when renewing. You can also create symbolic links, this will make your life easier but it’s less secure.

Configuring relayd

It’s time to configure relayd. The file is /etc/relayd.conf, and here’s an example configuration:

log state changes
prefork 10

table <itnotes> { 127.0.0.1 }

http protocol "http" {
    match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
    match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

    match request path "/*.atom" tag "CACHE"
    match request path "/*.css" tag "CACHE"
    match request path "/*.gif" tag "CACHE"
    match request path "/*.html" tag "CACHE"
    match request path "/*.ico" tag "CACHE"
    match request path "/*.jpg" tag "CACHE"
    match request path "/*.webp" tag "CACHE"
    match request path "/*.js" tag "CACHE"
    match request path "/*.png" tag "CACHE"
    match request path "/*.rss" tag "CACHE"
    match request path "/*.svg" tag "CACHE"
    match request path "/*.xml" tag "CACHE"
    match request path "/*.ttf" tag "CACHE"
    match request path "/*.woff2" tag "CACHE"

    match response tagged "CACHE" header set "Cache-Control" value "public, max-age=604800"

    pass request header "Host" value "it-notes.dragas.net" forward to <itnotes>
}

http protocol "https" {
    match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
    match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

    match response header set "Referrer-Policy" value "no-referrer"
    match response header set "X-Content-Type-Options" value "nosniff"
    match response header set "X-Download-Options" value "noopen"
    match response header set "X-Frame-Options" value "SAMEORIGIN"
    match response header set "X-Permitted-Cross-Domain-Policies" value "none"
    match response header set "X-XSS-Protection" value "1; mode=block"
    match response header set "Strict-Transport-Security" value "max-age=15552000; includeSubDomains; preload"

    match request path "/*.atom" tag "CACHE"
    match request path "/*.css" tag "CACHE"
    match request path "/*.gif" tag "CACHE"
    match request path "/*.html" tag "CACHE"
    match request path "/*.ico" tag "CACHE"
    match request path "/*.jpg" tag "CACHE"
    match request path "/*.webp" tag "CACHE"
    match request path "/*.js" tag "CACHE"
    match request path "/*.png" tag "CACHE"
    match request path "/*.rss" tag "CACHE"
    match request path "/*.svg" tag "CACHE"
    match request path "/*.xml" tag "CACHE"
    match request path "/*.ttf" tag "CACHE"
    match request path "/*.woff2" tag "CACHE"

    match response tagged "CACHE" header set "Cache-Control" value "public, max-age=604800"
    tcp { nodelay, sack, socket buffer 65536, backlog 100 }

    tls { keypair "it-notes.dragas.net" }

    pass request header "Host" value "it-notes.dragas.net" forward to <itnotes>
}

relay "http" {
    listen on vio0 port 80
    protocol "http"

    forward to <itnotes> port 8080
}

relay "https" {
    listen on vio0 port 443 tls
    protocol "https"

    forward to <itnotes> port 8080
}

relay "https6" {
    listen on my:ip:v6:address::1 port 443 tls
    protocol "https"

    forward to <itnotes> port 8080
}

The content of the file is quite self-explanatory. Note that:

vio0 is the interface name—it should be modified based on the interface where relayd needs to listen.
I’ve configured relayd to listen on port 80 as well.
IPv4 and IPv6 listeners are separated. If IPv6 is not configured, simply comment out that part. Please, if you can, use IPv6. In a fairer world, everyone would have the right to at least one class of public addresses without having to pay exorbitant fees.
The "keypair" must correspond to the certificate and key names in /etc/ssl and /etc/ssl/private.

Test the configuration, enable, and start relayd:

obcdn# relayd -n
configuration OK
obcdn# rcctl enable relayd
obcdn# rcctl start relayd
relayd(ok)

Final Checks

The stack is ready. A ps command will show the process status:

obcdn# ps aux
USER       PID %CPU %MEM   VSZ   RSS TT  STAT   STARTED       TIME COMMAND
root         1  0.0  0.0   920   276 ??  I       7:10PM    0:00.01 /sbin/init
[...]
_varnish 44999  0.0  0.2  2256  2424 ??  S       8:27PM    0:00.05 varnishd: Varnish-Mgt -i obcdn.my.domain (varnishd)
_varnish 54481  0.0  8.8 34500 88876 ??  S       8:27PM    0:00.60 varnishd: Varnish-Child -i obcdn.my.domain (varnishd)
root     57753  0.0  0.5  4080  4820 ??  IU      8:32PM    0:00.03 /usr/sbin/relayd
_relayd  95940  0.0  0.4  2152  3892 ??  Spc     8:32PM    0:00.01 relayd: pfe (relayd)
_relayd  81032  0.0  0.4  2156  3716 ??  Spc     8:32PM    0:00.01 relayd: hce (relayd)
_relayd  80312  0.0  0.5  2748  5280 ??  Ipc     8:32PM    0:00.03 relayd: relay (relayd)
_relayd  88919  0.0  0.5  2748  5268 ??  Ipc     8:32PM    0:00.04 relayd: relay (relayd)
_relayd  90186  0.0  0.5  2752  5272 ??  Ipc     8:32PM    0:00.03 relayd: relay (relayd)
_relayd  32851  0.0  0.5  2736  5284 ??  Ipc     8:32PM    0:00.03 relayd: relay (relayd)
_relayd  20270  0.0  0.5  2744  5252 ??  Ipc     8:32PM    0:00.03 relayd: relay (relayd)
_relayd  98826  0.0  0.5  2752  5264 ??  Ipc     8:32PM    0:00.04 relayd: relay (relayd)
_relayd   6454  0.0  0.5  2744  5264 ??  Ipc     8:32PM    0:00.03 relayd: relay (relayd)
_relayd  90102  0.0  0.5  2740  5276 ??  Ipc     8:32PM    0:00.03 relayd: relay (relayd)
_relayd  60314  0.0  0.5  2748  5336 ??  Ipc     8:32PM    0:00.03 relayd: relay (relayd)
_relayd  27246  0.0  0.5  2744  5284 ??  Ipc     8:32PM    0:00.03 relayd: relay (relayd)
_relayd  87082  0.0  0.4  2100  4508 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  53308  0.0  0.4  2096  4496 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  11697  0.0  0.4  2092  4492 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  83636  0.0  0.4  2092  4500 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  74563  0.0  0.4  2096  4484 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  35910  0.0  0.4  2100  4508 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  24911  0.0  0.4  2096  4504 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  78649  0.0  0.4  2096  4496 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  89586  0.0  0.4  2096  4500 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
_relayd  11989  0.0  0.4  2100  4500 ??  Ipc     8:32PM    0:00.02 relayd: ca (relayd)
[...]

In this case, both relayd and Varnish are running correctly.

Automating Certificate Renewal

As a final step, remember to create a script to renew the certificates, copy them to /etc/ssl and /etc/ssl/private, and restart relayd.

Conclusion

Congratulations, you now have your own free CDN, with your data, fully under your control. Portable, extendable, controllable, and outside of the big providers' grip.

If your goal is geo-replication, you should use one of the available methods. Some DNS providers allow selection based on the caller's location (like Bunny.net), or, as I prefer, install your own DNS and use tools to manage operations and resolutions, as briefly described in the previous article.

With multiple separate reverse proxies, separate DNS servers (on different providers and possibly different countries or continents) capable of checking if the reverse proxies are operational, you can achieve an extremely low likelihood of encountering a Single Point of Failure, as all components, once the cache is filled, will be nearly autonomous even in the event of a (temporary) backend outage - i.e., the original node.

Make your own VPN - OpenBSD, Wireguard, ipv6 and ad-blocking included

Stefano Marinelli — Mon, 03 Apr 2023 04:16:11 +0000

Note: This article assumes a setup based on OpenBSD. If you prefer a version based on FreeBSD, it is available here.

VPNs are a fundamental tool for securely connecting to your own servers and devices. Many people use commercial VPNs for various reasons, ranging from not trusting their provider (especially when connecting from a public hotspot) to wanting to "go out" on the Internet with a different IP address, perhaps from another country.

Whatever the reason, solutions are not lacking. I have always set up management VPNs to allow servers and/or clients to communicate with each other using secure channels. Lately, I have been activating IPv6 connectivity on all my devices (both desktop/servers and mobile devices) and I needed to quickly create a node that concentrated some networks and allowed them to go out on the network in IPv6. The tools I used and will describe are:

VPS - in this case, I used a basic Hetzner Cloud VPS (using this link, you will receive 20 euros of cloud credits), but any provider that provides IPv6 connectivity will do - if you want IPv6, of course.
OpenBSD - a clean, stable, and secure operating system.
Wireguard - lightweight, secure, and at the same time, not very "chatty", so it is also gentle on mobile device batteries. When there is no traffic, it simply does not transmit/receive anything. Well supported by all major desktop and server operating systems as well as Android and iOS devices.
Unbound - can make DNS queries directly to root servers, not through forwarders. It also allows you to insert block-lists and have a result similar to that of Pi-Hole (i.e., ad-blocking).
SpamHaus lists - to immediately stop connections to and from users on blacklists.

The first step is to activate a VPS and install OpenBSD. On the Hetzner cloud console, there won't be a pre-built OpenBSD image, but only a selection of Linux distributions. Don't worry, just choose any of them and create the VPS. Once done, the OpenBSD ISO image will be available among the "ISO Images". Just insert the virtual CD, restart the VPS, and the OpenBSD installation will appear in the console.

I won't go into detail, the operation is simple and straightforward. The only precaution (in the case of a Hetzner Cloud VPS) is to use "autoconf" for IPv4 but, for now, do not configure IPv6. It will be configured later.

Install all OpenBSD updates (using the syspatch command) and restart, the kernel will be relinked.

Wireguard, on OpenBSD, is fully integrated into the base system and does not require the installation of external packages. This is a big advantage because over time, support for everything related to Wireguard will be managed directly by the main OpenBSD development team.

The first step is to configure IPv6 on the VPS. In the case of Hetzner, unfortunately, they only provide a /64, so it will be necessary to segment the assigned network. In this example, it will be divided into /72 subnetworks - to find valid subclasses, it will be possible to use a calculator.

The /etc/hostname.vio0 file should look something like this:

inet autoconf
inet6 2001:db8:cafe:cafe::1 72 
!route add -net ::/0 fe80::1%vio0

In short, keep the base address assigned by Hetzner, but change the netmask to /72 - thus giving the possibility of having other networks available.

sh /etc/netstart vio0

It will reconfigure the network interface and allow IPv6 to start working. To test it:

ping6 google.com

If everything has been configured correctly, the ping will be executed and google.com will reply.

It is now necessary to enable forwarding for IPv4 and IPv6. Enter these lines in the /etc/sysctl.conf file:

net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1

To apply those changes you can reboot or just type:

sysctl net.inet.ip.forwarding=1
sysctl net.inet6.ip6.forwarding=1

To configure Wireguard, a few steps will be necessary. First of all, the private key will need to be created:

openssl rand -base64 32

Something like this will come out:

YUkS6cNTyPbXmtVf/23ppVW3gX2hZIBzlHtXNFRp80w=

Now create a new file called /etc/hostname.wg0:

172.16.0.1/24 wgport 51820 wgkey YUkS6cNTyPbXmtVf/23ppVW3gX2hZIBzlHtXNFRp80w=
inet6 2001:db8:cafe:cafe:100::1 72

A new Wireguard interface called wg0 is being created. It will have the IPv4 address "172.16.0.1", Wireguard will listen on port 51820, and with the private key created shortly before. It will also have an IPv6 address on one of the subclasses that the provider will have provided.

Save and activate the interface:

sh /etc/netstart wg0

If everything has been entered correctly, it should enable the interface. Now:

ifconfig wg0

And something like this will be returned:

wg0: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1420
    index 5 priority 0 llprio 3
    wgport 51820
    wgpubkey xxxxxxxxxxxxxxx=
    groups: wg
    inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255
    inet6 2001:db8:cafe:cafe:100::1 prefixlen 72

Take note of the wgpubkey - it will be needed to configure the clients.

As for the firewall, OpenBSD comes with a basic pf configuration. In my setups, I tend to block what is not needed and be permissive with what may be useful. However, I like to keep out the "bad guys," so I use blacklists. pf allows elements to be inserted and removed from tables in runtime, so the firewall can be configured accordingly.

To download and apply Spamhaus lists, I use a simple but effective script found on the Internet.

So create the script in /usr/local/sbin/spamhaus.sh:

#!/bin/sh
#
# this is normally run once per day via /etc/daily.local.
#
echo updating Spamhaus DROP lists:
(
  { ftp -o - https://www.spamhaus.org/drop/drop.txt && \
    ftp -o - https://www.spamhaus.org/drop/dropv6.txt ; \
  } 2>/dev/null | sed "s/;/#/" > /var/db/drop.txt
)
pfctl -t spamhaus -T replace -f /var/db/drop.txt

Make it executable and run it:

chmod a+rx /usr/local/sbin/spamhaus.sh
/usr/local/sbin/spamhaus.sh

There are many possibilities to configure pf. A fairly simple example could be this:

ext_if="vio0"
wg0_if = "wg0"
wg0_networks = "172.16.0.0/24"

set skip on lo

# Spamhaus DROP list:
table <spamhaus> persist file "/var/db/drop.txt"

block drop log quick from <spamhaus>

match in all scrub (no-df random-id max-mss 1440)

match out on $ext_if from { $wg0_networks } nat-to ($ext_if)

#Pass ICMP on ipv6
pass quick proto ipv6-icmp
#Block from ipv6 to wg0 network
block in quick on $ext_if inet6 to { 2001:db8:cafe:cafe:100::/72 }
#Pass Wireguard traffic - in and out
pass quick on $wg0_if

# default deny
block in
block out

# By default, do not permit remote connections to X11
block return in on ! lo0 proto tcp to port 6000:6010

# Port build user does not need network
block return out log proto {tcp udp} user _pbuild

pass in on $ext_if proto tcp to port ssh
pass in on $ext_if proto udp to port 51820

pass out on $ext_if

This is a very simple configuration: it blocks everything that is present in the list downloaded from Spamhaus, allows NAT from the Wireguard network to the public interface, allows icmp traffic in IPv6 (necessary for the network to function properly) while blocking incoming traffic to the Wireguard IPv6 LAN (remember that the IPs will be public and directly reachable, so we don't want to expose our devices by default). All traffic on the Wireguard interface will be allowed to pass. Then everything will be blocked and exceptions will be specified, i.e. allowing ssh and Wireguard connections (of course). Authorization will also be granted to allow traffic to exit from the public network interface.

Reload pf configuration:

pfctl -f /etc/pf.conf

If everything went correctly, the firewall should have loaded the new options.

To obtain caching of DNS queries and the related ad-block, it is now time to configure Unbound. A while ago, I found a script which I slightly adapted. I don't remember where I got it, so I'll paste it here without citing the original creator.

Create a script to update the unbound ad-block, in /usr/local/sbin/unbound-adhosts.sh:

#!/bin/ksh
#
# Using blacklist from pi-hole project https://github.com/pi-hole/
# to enable AD blocking in unbound(8)
#
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

# Available blocklists - comment line to disable blocklist
_disconad="https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt"
_discontrack="https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt"
_stevenblack="https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts"

# Global variables
_tmpfile="$(mktemp)" && echo '' > $_tmpfile
_unboundconf="/var/unbound/etc/unbound-adhosts.conf"

# Remove comments from blocklist
function simpleParse {
  ftp -VMo - $1 | \
  sed -e 's/#.*$//' -e '/^[[:space:]]*$/d' >> $2
}

# Parse MalwareDom
#[[ -n ${_malwaredom+x} ]] && simpleParse $_malwaredom $_tmpfile

# Parse ZeusTracker
#[[ -n ${_zeustracker+x} ]] && simpleParse $_zeustracker $_tmpfile

# Parse DisconTrack
[[ -n ${_discontrack+x} ]] && simpleParse $_discontrack $_tmpfile

# Parse DisconAD
[[ -n ${_disconad+x} ]] &&  simpleParse $_disconad $_tmpfile

# Parse StevenBlack
[[ -n ${_stevenblack+x} ]] && \
  ftp -VMo - $_stevenblack | \
  sed -n '/Start/,$p' | \
  sed -e 's/#.*$//' -e '/^[[:space:]]*$/d' | \
  awk '/^0.0.0.0/ { print $2 }' >> $_tmpfile

# Parse hpHosts
[[ -n ${_hostfiles+x} ]] && \
  ftp -VMo - $_hostfiles | \
  sed -n '/START/,$p' | tr -d '^M$' | \
  sed -e 's/#.*$//' -e '/^[[:space:]]*$/d' -e 's/^M$//' | \
  awk '/^127.0.0.1/ { print $2 }' >> $_tmpfile

# Create unbound(8) local zone file
sort -fu $_tmpfile | grep -v "^[[:space:]]*$" | \
awk '{
  print "local-zone: \"" $1 "\" redirect"
  print "local-data: \"" $1 " A 0.0.0.0\""
}' > $_unboundconf && rm -f $_tmpfile

/usr/sbin/rcctl reload unbound 1>/dev/null

exit 0
#EOF

Similarly, make the script executable and run it:

chmod a+rx /usr/local/sbin/unbound-adhosts.sh
/usr/local/sbin/unbound-adhosts.sh

Now, the Unbound configuration file in /var/unbound/etc/unbound.conf can be modified as follows:

# $OpenBSD: unbound.conf,v 1.21 2020/10/28 11:35:58 sthen Exp $

server:
        verbosity: 1
        log-queries: no
        num-threads: 4
        num-queries-per-thread: 1024
        interface: 127.0.0.1
        #interface: 127.0.0.1@5353      # listen on alternative port
        interface: 172.16.0.1
    interface: 2001:db8:cafe:cafe:100::1
        interface: ::1
        outgoing-range: 64
        chroot: ""
        #do-ip6: yes

        # override the default "any" address to send queries; if multiple
        # addresses are available, they are used randomly to counter spoofing
        #outgoing-interface: 192.0.2.1
        #outgoing-interface: 2001:db8::53

        access-control: 0.0.0.0/0 refuse
        access-control: 127.0.0.0/8 allow
        access-control: ::0/0 refuse
        access-control: ::1 allow
        access-control: 172.16.0.0/24 allow
        access-control: 2001:db8:cafe:cafe:100::/72 allow

        hide-identity: yes
        hide-version: yes

        # Perform DNSSEC validation.
        #
        auto-trust-anchor-file: "/var/unbound/db/root.key"
        val-log-level: 2

        # Synthesize NXDOMAINs from DNSSEC NSEC chains.
        # https://tools.ietf.org/html/rfc8198
        #
        aggressive-nsec: yes
        prefetch: yes
        username: "nobody"
        directory: "/var/unbound/etc"
        logfile: "/var/unbound/unbound.log"
        use-syslog: no
        pidfile: "/var/unbound/unbound.pid"
        include: /var/unbound/etc/unbound-adhosts.conf

remote-control:
        control-enable: yes
        control-interface: /var/run/unbound.sock

Before launching unbound, it is necessary to give the appropriate permissions:

chown -R nobody:nobody /var/unbound

Now it is possible to enable and start unbound. Since it needs to load the (long) blocklist, it will take a few seconds:

rcctl enable unbound
rcctl start unbound

If everything has been done correctly, unbound will be able to respond to requests made on 172.16.0.1 and 2001:db8:cafe:cafe:100::1, from their respective LANs.

Now it is possible to configure the Wireguard client. Each implementation has its own procedure (Android, iOS, MikroTik, Linux, etc.) but essentially it is sufficient to create the right configuration both on the server and on the client. For example, the server's public key (visible by typing "ifconfig wg0" on the OpenBSD server) should be inserted into the "peer" that will be created on the client, while the client's public key will be used on the server in this way:

Reopen the file /etc/hostname.wg0 and add:

172.16.0.1/24 wgport 51820 wgkey YUkS6cNTyPbXmtVf/23ppVW3gX2hZIBzlHtXNFRp80w=
inet6 2001:db8:cafe:cafe:100::1 72
wgpeer *client's public key* wgaip 172.16.0.2/32 wgaip 2001:db8:cafe:cafe:100::2/128

Reload the configuration:

sh /etc/netstart wg0

On the client, create a new configuration by inserting "172.16.0.2/32, 2001:db8:cafe:cafe:100::2/128" (the ones that were entered in the peer configuration in the hostname.wg0 file) in the local IP addresses. Set the DNS server address to "172.16.0.1" and/or its corresponding IPv6 address (in the example, 2001:db8:cafe:cafe:100::1 - yours will be different). In the peer, insert the server's data, including its public key, IP address:port (in the example, the port is 51820), and allowed addresses (setting "0.0.0.0/0, ::0/0" means "all connections will be sent via Wireguard" - all the traffic will pass through the VPN for both IPv4 and IPv6).

It is also possible to use the VPN only as an ad-blocker, by only routing DNS traffic through it. To achieve this result, it is sufficient to configure the client so that the only allowed address is the one of the just-configured unbound (in this example, 172.16.0.1 or 2001:db8:cafe:cafe:100::1) - DNS resolution will occur via VPN, but browsing will continue to work through the main provider.

If you want the spamhaus and ad-block lists to be updated automatically, create the /etc/daily.local file and add the following lines:

#!/bin/ksh

/usr/local/sbin/unbound-adhosts.sh
/usr/local/sbin/spamhaus.sh

All of this can be achieved simply with a basic installation of OpenBSD, without the need to install any additional packages. This is an advantage both in terms of update management and security.