Proxmox vs FreeBSD: Which Virtualization Host Performs Better?
Since migrating many servers from Proxmox to FreeBSD, we have consistently felt that the VMs are more responsive. It's time to conduct some concrete tests.
https://it-notes.dragas.net/2024/06/10/proxmox-vs-freebsd-which-virtualization-host-performs-better/
#FreeBSD #Proxmox #Linux #Virtualization #kvm #bhyve #IT #SysAdmin #ITNotes #NoteHUB
Proxmox vs FreeBSD: Which Virtualization Host Performs Better?
Comparative tests show FreeBSD's virtualization performance surpasses Proxmox, especially with NVMe drivers.Stefano Marinelli (it-notes.dragas.net)
❄️ freezr ❄️ likes this.
reshared this
moozer
in reply to Stefano Marinelli • • •This is interesting. Thanks.
The file creation speed is very significant - going from 407 MB/s on proxmox to 1467 MB/s on freebsd.
Do you know why there is such a large difference?
Stefano Marinelli
in reply to moozer • • •cuddle
in reply to Stefano Marinelli • • •ZFS on Linux still not mature yet, and FreeBSD support ZFS out of the box, to the kernel level.
If ZFS had close integration with the Linux kernel, it would have much or less similar I/O performance, both are POSIX complaint systems after all.
Paul Wilde :dontpanic2: :smeghead:
in reply to Stefano Marinelli • • •Thanks,
I've recently moved some of my VMs from KVM to bhyve and have noticed a significant performance improvement.
Most noticeably with Nextcloud (Debian guest) where it feels really quick and snappy now where it felt clunky before. I guess this may be due to improved IO performance similar to what you have tested.
Stefano Marinelli
in reply to Paul Wilde :dontpanic2: :smeghead: • • •Benjamin Kwiecień 🇵🇸
in reply to Paul Wilde :dontpanic2: :smeghead: • • •Paul Wilde :dontpanic2: :smeghead:
in reply to Benjamin Kwiecień 🇵🇸 • • •to be fair it's probably ZFS doing a lot of the heavy lifting here, but @stefano 's testing has shown that even on ZFS on proxmox (KVM) the performance still isn't quite as good as ZFS on FreeBSD (bhyve) - which is likely due to the ZFS modules in FreeBSD being native over additional kernel modules in proxmox (or any other Linux distro that can use it)
bhyve is definitely worth looking into if you use VMs though, for sure!
Phil Dennis-Jordan 😷
in reply to Paul Wilde :dontpanic2: :smeghead: • • •BitPirate
in reply to Stefano Marinelli • • •Rule of thumb: if ZFS consistently beats ext4 in a basic filesystem benchmark on Linux, there's either something wrong with your setup or your benchmark is flawed.
I'd switch from sysbench to fio for I/O related tests.
Stefano Marinelli
in reply to BitPirate • • •BitPirate
in reply to Stefano Marinelli • • •I learned something new today and remembered our previous conversation. ext4 uses lazy initialization by default.
https://fedetft.wordpress.com/2022/01/23/on-ext4-and-forcing-the-completion-of-lazy-initialization/
On ext4 and forcing the completion of lazy initialization
fede.tft's Blogoxy
in reply to Stefano Marinelli • • •nice.
The results are interesting - I wouldnt have expected such a wide margin in some of the results.
That said, Ive never used Proxmox and it has been a while since I’ve used KVM at all (and at the time I was using Fedora/Centos on XFS - ext4 has eaten enough data for me to never ever want to use it again)
Stefano Marinelli
Unknown parent • • •Stefano Marinelli
Unknown parent • • •Stefano Marinelli
Unknown parent • • •Stefano Marinelli
Unknown parent • • •Fritz Adalis
in reply to Stefano Marinelli • • •Stefano Marinelli
in reply to Fritz Adalis • • •❄️ freezr ❄️
in reply to Stefano Marinelli • •@Stefano Marinelli
Nice article! 💪
Are you trying to do the competition to Phoronix? 🤪
like this
Stefano Marinelli and Stefano BSD Cafe (snac instance account) like this.
Stefano Marinelli
in reply to ❄️ freezr ❄️ • • •❄️ freezr ❄️ likes this.
Poul-Henning Kamp
in reply to Stefano Marinelli • • •Please next time you do benchmarks, use FreeBSD's ministat(1)