While I love FreeBSD-based router solutions (OPNsense, PFsense, etc.), I also appreciate and use MikroTik devices and software. They produce and sell reasonably-priced, efficient hardware, and have implemented some interesting proprietary solutions (EoIP, etc.). Additionally, they're a European company. From the smallest Wi-Fi router to the largest enterprise routing platform, they employ the same software with (almost) the same features.
That's why I'm also implementing virtualized MikroTik CHR solutions - they're lightweight, efficient, and can handle a significant amount of traffic with minimal resource overhead.
Our FreeBSD hypervisors run vm-bhyve as a lightweight, efficient, and intelligent management tool for VMs.
Even though MikroTik states that bhyve is not supported (as they believe it's merely a paravirtualization software), CHR based on RouterOS 6 works flawlessly, following the hints provided by the vm-bhyve documentation.
Unfortunately, things change when dealing with RouterOS 7. It doesn't boot at all, and if you upgrade from RoS 6 to RoS 7, it ceases to function entirely.
It seems that the MikroTik CHR image based on RouterOS 7 has an unusual partition table, somewhere between MBR and GPT. For this reason, Proxmox (KVM) doesn't experience any issues, while bhyve seems unable to properly boot the VM. I conducted several tests, but the best solution was suggested by user kriszos on the MikroTik forum (even if kriszos was dealing with Hyper-V).
The script should be used on a machine with the following tools installed: gdisk, wget, unzip, qemu-img, qemu-nbd, and rsync:
#!/bin/bash wget --no-check-certificate https://download.mikrotik.com/routeros/7.8/chr-7.8.img.zip -O /tmp/chr.img.zip unzip -p /tmp/chr.img.zip > /tmp/chr.img rm -rf chr.qcow2 qemu-img convert -f raw -O qcow2 /tmp/chr.img chr.qcow2 rm -rf /tmp/chr.im* modprobe nbd qemu-nbd -c /dev/nbd0 chr.qcow2 rm -rf /tmp/tmp* mkdir /tmp/tmpmount/ mkdir /tmp/tmpefipart/ mount /dev/nbd0p1 /tmp/tmpmount/ rsync -a /tmp/tmpmount/ /tmp/tmpefipart/ umount /dev/nbd0p1 mkfs -t fat /dev/nbd0p1 mount /dev/nbd0p1 /tmp/tmpmount/ rsync -a /tmp/tmpefipart/ /tmp/tmpmount/ umount /dev/nbd0p1 rm -rf /tmp/tmp* ( echo 2 # use GPT echo t # change partition code echo 1 # select first partition echo 8300 # change code to Linux filesystem 8300 echo r # Recovery/transformation echo h # Hybrid MBR echo 1 2 # partitions added to the hybrid MBR echo n # Place EFI GPT (0xEE) partition first in MBR (good for GRUB)? (Y/N) echo # Enter an MBR hex code (default 83) echo y # Set the bootable flag? (Y/N) echo # Enter an MBR hex code (default 83) echo n # Set the bootable flag? (Y/N) echo n # Unused partition space(s) found. Use one to protect more partitions? (Y/N) echo w # write changes to disk echo y # confirm ) | gdisk /dev/nbd0 qemu-nbd -d /dev/nbd0 echo "script finished, created file chr.qcow2"
I used the script on a machine with Proxmox installed, and it worked correctly. Once the relevant image was obtained, it was converted to raw format to be fed to bhyve:
qemu-img convert -f qcow2 -O raw chr.qcow2 chr.img
Now that we have the correct image available, here's an example of a VM configuration managed by vm-bhyve:
loader="uefi" graphics="no" cpu="2" memory=“128M" network0_type="virtio-net" network0_switch="public" disk0_type="virtio-blk" disk0_name=“chr.img” uuid=“cafecafe-cafe-cafe-cafe-cafecafecafe” network0_mac=“ca:fe:ca:fe:ca:fe”