Bridging Networks Across VPS With Wireguard and VXLAN on FreeBSD
![/featured/server_rack.webp](/featured/server_rack.webp)
Introduction
In today’s interconnected world, system administrators often face the challenge of managing services across multiple Virtual Private Servers (VPS). This article describes an advanced networking setup that allows you to bridge networks between two VPS instances using Wireguard and VXLAN on FreeBSD. This configuration is particularly useful when you need to distribute services across different providers or when you want to leverage the strengths of multiple hosting environments.
Background
At BSD Cafe, we utilize various VPS instances to provide our services. The two main ones are:
- A publicly accessible VPS that hosts the reverse proxy and all firewall rules for packet routing.
- A larger VPS on a physical host I own, which is not directly exposed to the internet and doesn’t have a public IP address.
Most of the jails hosting BSD Cafe services are distributed between these two VPS instances. Occasionally, I need to move services between them for performance reasons or to manage updates efficiently.
To facilitate this flexibility, I’ve always maintained a bridge on each VPS. Initially, I used Zerotier to establish a connection between these bridges, allowing them to communicate as if they were part of a single, large network.
The New Setup: Wireguard and VXLAN
While the Zerotier setup worked, I decided to switch to a more streamlined solution using Wireguard and VXLAN. Here’s why:
- Performance: Wireguard offers excellent performance with low overhead.
- Simplicity: The configuration is straightforward and easy to maintain.
- Security: Wireguard provides strong, modern cryptography.
I had already prepared a Wireguard connection between the two servers from the beginning. Since only one of the servers is publicly accessible, I set up one to only accept connections and the other to connect directly to the public IP of the first, with a 20-second keepalive (which is generally not necessary due to the high traffic between the jails).
To complete the setup, I added two VXLAN interfaces on the VPS instances, added these interfaces to the local bridges, and immediately, packets started flowing between the networks.
Overall Network Topology
graph TB subgraph "Public Network" Internet((Internet)) Internet2((Internet)) end subgraph "VPS 1 (Public IP)" PublicIP[Public IP] Firewall[Firewall] ReverseProxy[Reverse Proxy] Bridge1[Bridge] WG1[Wireguard] VXLAN1[VXLAN] Jail1[Jail 1] Jail2[Jail 2] end subgraph "VPS 2 (Behind NAT)" NatDev2[Nat] Firewall2[Firewall] Bridge2[Bridge] WG2[Wireguard] VXLAN2[VXLAN] Jail3[Jail 3] Jail4[Jail 4] end Internet <--> PublicIP PublicIP <--> Firewall Internet2 --- NatDev2 NatDev2 <--> Firewall2 Firewall2 <--> WG2 Firewall <--> WG1 Firewall --> ReverseProxy ReverseProxy --> Bridge1 Bridge1 --- Jail1 Bridge1 --- Jail2 Bridge1 --- VXLAN1 VXLAN1 <---> WG1 Bridge2 --- Jail3 Bridge2 --- Jail4 Bridge2 --- VXLAN2 VXLAN2 <---> WG2 Internet <--> Internet2graph TB subgraph "Public Network" Internet((Internet)) Internet2((Internet)) end subgraph "VPS 1 (Public IP)" PublicIP[Public IP] Firewall[Firewall] ReverseProxy[Reverse Proxy] Bridge1[Bridge] WG1[Wireguard] VXLAN1[VXLAN] Jail1[Jail 1] Jail2[Jail 2] end subgraph "VPS 2 (Behind NAT)" NatDev2[Nat] Firewall2[Firewall] Bridge2[Bridge] WG2[Wireguard] VXLAN2[VXLAN] Jail3[Jail 3] Jail4[Jail 4] end Internet <--> PublicIP PublicIP <--> Firewall Internet2 --- NatDev2 NatDev2 <--> Firewall2 Firewall2 <--> WG2 Firewall <--> WG1 Firewall --> ReverseProxy ReverseProxy --> Bridge1 Bridge1 --- Jail1 Bridge1 --- Jail2 Bridge1 --- VXLAN1 VXLAN1 <---> WG1 Bridge2 --- Jail3 Bridge2 --- Jail4 Bridge2 --- VXLAN2 VXLAN2 <---> WG2 Internet <--> Internet2
Step-by-Step Implementation
Follow these instructions to create a bridge between two different networks using Wireguard and VXLAN on FreeBSD. While I use this setup to connect jails at BSD Cafe, you can use it for various purposes, such as bridging different VM (bhyve) instances across providers.
Prerequisites
Wireguard is now an integral part of FreeBSD, so you no longer need to compile a module or use the Go version. However, we’ll use the “wireguard-tools” scripts as they provide the useful “wg-quick” command.
Start by installing the wireguard-tools package on both servers:
|
|
Configuration
Server 1 (Public IP)
- Generate the Wireguard keys:
|
|
This command will output a private key and a public key. Note down the public key as you’ll need it to configure the client.
Let’s also add a PSK; it’s optional but will increase the security of the entire setup.
|
|
- Create a new file
/usr/local/etc/wireguard/wg0.conf
:
|
|
- Modify
/etc/rc.conf
and add:
|
|
- Start Wireguard and the VXLAN endpoint:
|
|
Server 2 (Behind NAT)
Generate the Wireguard keys as before.
Create
/usr/local/etc/wireguard/wg0.conf
:
|
|
Modify
/etc/rc.conf
as before.Start Wireguard and the VXLAN endpoint:
|
|
Verifying the Connection
To check if the connection is established, run the wg
command on either host. This will show you the connection status, the last handshake, and the data transferred.
You can also try pinging the other host’s Wireguard and VXLan interface IP address (in this example, 10.77.0.1 or 10.77.0.2 and 10.77.1.1 or 10.77.1.2).
VXLAN Configuration Detail
graph LR subgraph "VPS 1 (10.77.0.1)" Bridge1[Bridge0] VXLAN1[VXLAN0\n10.77.1.1/24] WG1[Wireguard\nwg0] end subgraph "VPS 2 (10.77.0.2)" Bridge2[Bridge0] VXLAN2[VXLAN0\n10.77.1.2/24] WG2[Wireguard\nwg0] end Bridge1 --- VXLAN1 Bridge2 --- VXLAN2 VXLAN1 -.- WG1 VXLAN2 -.- WG2 WG1 <--> |Encrypted Tunnel| WG2 classDef vxlan fill:#f9f,stroke:#333,stroke-width:4px; class VXLAN1,VXLAN2 vxlan;graph LR subgraph "VPS 1 (10.77.0.1)" Bridge1[Bridge0] VXLAN1[VXLAN0\n10.77.1.1/24] WG1[Wireguard\nwg0] end subgraph "VPS 2 (10.77.0.2)" Bridge2[Bridge0] VXLAN2[VXLAN0\n10.77.1.2/24] WG2[Wireguard\nwg0] end Bridge1 --- VXLAN1 Bridge2 --- VXLAN2 VXLAN1 -.- WG1 VXLAN2 -.- WG2 WG1 <--> |Encrypted Tunnel| WG2 classDef vxlan fill:#f9f,stroke:#333,stroke-width:4px; class VXLAN1,VXLAN2 vxlan;
Conclusion
This setup allows the two VXLAN interfaces, inserted into the local bridge, to enable packet transit through Wireguard. This facilitates free passage between the two hosts, effectively creating a single, unified network across your VPS instances.
This configuration is particularly useful for:
- Distributing services across different providers
- Leveraging both public-facing and private VPS instances
- Creating flexible, scalable network architectures
By using Wireguard and VXLAN, you get the benefits of strong encryption, high performance, and the ability to create complex network topologies across physically separate servers.
Remember to always keep your systems updated and regularly review your network configuration to ensure it meets your evolving needs and security requirements.