Moving an entire FreeBSD installation to a new host or VM in a few easy steps

FreeBSD, especially when installed on ZFS, is incredibly simple to manage, back up, and move to a different system.

I often find myself having to move an entire system from one host to another, from a physical host to a VM, or vice versa, from a VM to a physical host.

By following the approach of keeping the operating system clean and running all services within jails, this operation is usually quite simple: install the operating system on the new host, perform the few necessary configurations (like pf and network interfaces), transfer the jail datasets, restart the jails on the new host, and update the DNS.

However, sometimes I need to move the entire host, including the original operating system. This article will describe this operation when the FreeBSD system uses ZFS as its root file system. This process isn't much more difficult, but there are a few details to keep in mind. Below is a breakdown of the two bootloader scenarios:

• UEFI Bootloader
• BIOS Bootloader (non-UEFI)

UEFI Bootloader

• Start from an ISO or image of mfsbsd. Partition the disk according to the layout of the source system. If the source system is a standard FreeBSD (ZFS) installation, partition the destination disk (assumed to be da0) as follows. WARNING: The first command will destroy any existing partition table on the disk.

gpart destroy -F /dev/da0
gpart create -s gpt da0
gpart add -t efi -s 200M -l efiboot0 da0
newfs_msdos -F 32 -c 1 /dev/da0p1
mount -t msdosfs /dev/da0p1 /mnt
mkdir -p /mnt/EFI/BOOT
cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi
umount /mnt

• Create additional partitions for swap and ZFS:

gpart add -a 1m -t freebsd-swap -s 2g -l swap0 da0
gpart add -a 1m -t freebsd-zfs -l zfs0 da0

• Create the ZFS pool:

zpool create -O compression=zstd -O atime=off zroot /dev/gpt/zfs0

• On the source system, create a snapshot of all datasets:

zfs snapshot -r zroot@snap01

• Important: You will need to connect to the source system using a user with privileges over the entire filesystem. For simplicity, I'll use root, but it's not recommended to leave root accessible via SSH. Limit access to private keys only. To allow root login over SSH, edit /etc/ssh/sshd_config and set PermitRootLogin yes. After moving the server, remember to disable root access.

• If mbuffer is installed on the source system, it can help improve the performance of the transfer. If not, remove it from the next pipe command, which should be run from the destination system. The -s will set the block-size, the -m will set the buffer size (128MB, here):

ssh root@sourceip "zfs send -RLv zroot@snap01 | mbuffer -s 128k -m 128M" | zfs receive -F zroot

• You have two options now:
• If the source server was idle and no further synchronization is required, proceed to the next step.
• If the transfer took a long time and services on the source server need to be stopped (e.g., databases), stop what you can, take another snapshot (e.g., zfs snapshot -r zroot@snap02), and synchronize the differences by running another incremental send-receive from the destination server:

ssh root@sourceip "zfs send -RLv -i zroot@snap01 zroot@snap02 | mbuffer -s 128k -m 128M" | zfs receive -F zroot

• Set the boot filesystem:

zpool set bootfs=zroot/ROOT/default zroot

BIOS Bootloader

• Start from an ISO or image of mfsbsd. Partition the disk according to the layout of the source system. If the source system is a standard FreeBSD (ZFS) installation, partition the destination disk (assumed to be da0) as follows. WARNING: The first command will destroy any existing partition table on the disk.

gpart destroy -F /dev/da0
gpart create -s gpt da0

• Create the necessary partitions:

gpart add -a 1m -t freebsd-boot -s 512k -l boot da0
gpart add -a 1m -t freebsd-swap -s 2g -l swap0 da0
gpart add -a 1m -t freebsd-zfs -l zfs0 da0

• Create the ZFS pool:

zpool create -O compression=zstd -O atime=off zroot /dev/gpt/zfs0

• On the source system, create a snapshot of all datasets:

zfs snapshot -r zroot@snap01

• Important: You will need to connect to the source system using a user with privileges over the entire filesystem. For simplicity, I'll use root, but it's not recommended to leave root accessible via SSH. Limit access to private keys only. To allow root login over SSH, edit /etc/ssh/sshd_config and set PermitRootLogin yes.

• If mbuffer is installed on the source system, it can help improve the performance of the transfer. If not, remove it from the next pipe command, which should be run from the destination system:

ssh root@sourceip "zfs send -RLv zroot@snap01 | mbuffer -s 128k -m 128M" | zfs receive -F zroot

• You have two options now:
• If the source server was idle and no further synchronization is required, proceed to the next step.
• If the transfer took a long time and services on the source server need to be stopped (e.g., databases), stop what you can, take another snapshot (e.g., zfs snapshot -r zroot@snap02), and synchronize the differences by running another incremental send-receive from the destination server:

ssh root@sourceip "zfs send -RLv -i zroot@snap01 zroot@snap02 | mbuffer -s 128k -m 128M" | zfs receive -F zroot

• Install the bootloader on the destination server’s disk:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

• Set the boot filesystem:

zpool set bootfs=zroot/ROOT/default zroot

At this point, everything should be ready for boot. Reboot and verify that the various mountpoints (e.g., swap, etc.) and network interface settings are correct. If everything works fine, the server will be an exact copy of the original.

Moving or duplicating an entire FreeBSD system is simple, fast, and reliable. The ability to use ZFS incremental replication is an excellent method for minimizing downtime, especially for large systems. The initial copy will happen "live" while the source system continues to run without issues. Subsequent incrementals will be much faster, making the switch-over process seamless and efficient.