If you have a home lab or considering to build one, this is or will become a very important decision. Once you have the hardware ready, it is time to choose a virtualization platform or hypervisor. Luckily there are a few to choose from.

Of course VMware is probably the first one that comes to mind, they have been in the virtualization business for many years and lead the way. VMware (and I carefully say this) used to always come out on top in comparison to other platforms, however Hyper-V and KVM have caught up pretty good in recent years.

It’s also important to understand that products like ESXi and Hyper-V offer a free version but are still proprietary software, and free version will have some limitations at least in the case of ESXi.

Let’s have a closer look at each one.

ESXi/vSphere:

The free version can get you started, all you need is a VMware account to get the license. Of course it has limitations:

  1. Max 2 physical CPU in host
  2. Max 8 vCPU for VMs
  3. Cannot be managed with vCenter
  4. Backup APIs locked - cannot use hypervisor level backup software like Veeam B&R, Vembu or (my favourite) Unitrends

Should I care about these limitations for home lab use?

  1. Physical CPU: Not an issue for home lab or even enterprise in most cases.
  2. vCPU: May be a deal breaker for some home labs.
  3. No vCentre: Lack of central management is understandable, because this is a proprietary software with licensing options and the free version is a good start but not intended for mission critical workloads. You can always manage the server via ESXi web client.
  4. No hypervisor level backup: This should be a deal breaker even for home lab use. You may not consider backups when you start with virtualization but as you spend time and build more on top of the platform, backup and recovery becomes very important. This also serves very well for the learning experience because you won’t find many (if any at all) organizations that do not implement a backup and recovery plan.

I would not recommend the free ESXi, simply because it is very limited and if you are after a free hypervisor then Hyper-V or Proxmox/KVM are the better choices.

For personal use or home lab, VMUG Advantage subscription is available for $200 USD per year. With that you get vSphere Enterprise licensing for 6 CPUs, vCenter, Workstation Pro and whole bunch of other VMware products.

Hyper-V Server

Microsoft offers a free version of their hypervisor. You won’t get a GUI of course but there are no hypervisor limitations. This means that Veeam, Unitrends and Vembu can perform a hypervisor level backup of your VMs. You can also perform live migration between Hyper-V hosts.

It sounds great but there’s a catch and for Hyper-V Server it is the prep work before you can use and manage it (without a domain). You will also need to be comfortable with PowerShell and command line at least to get things running. Unlike (free) ESXi there is no web client.

The easiest management tool, if you already have a machine with Windows 10 Pro/Ent, is the Hyper-V manager. You can also install and use Windows Admin Center to manage Hyper-V Server via web based interface, and finally PowerShell core is available on Linux so you can manage it from any Linux machine as well.

Hyper-V manager does a very good job in a home lab but of course with enterprise software, good central management comes with licensing and (huge) cost: SCVMM.

Hyper-V may not be as friendly as ESXi but does pack features like live migration and ability to backup without spending a single dime.

Proxmox/KVM

The solution to all the problems (mentioned above) and escape from proprietary software and enterprise licensing model: Proxmox and KVM.

If you have any doubts about KVM, you should know that AWS and Google use it.

KVM provides the hypervisor, QEMU takes care of hardware emulation and Proxmox adds the management layer and does an absolutely amazing job with that.

Since it is an open source software, you don’t need a license and all the features are available to you after installation. It includes built-in backup, replication, HA, distributed storage (ceph), local and remote storage options, live migration and containerization via LXC. There’s no need for a client software or separate management appliance, it’s all available on the web-based management and from any server in the cluster.

Proxmox also offer subscription based support plans and without one you just don’t get access to Enterprise repository that provides stable updates and enhanced security. You still get updates from the community channel for Proxmox.

This is a complete package without any limitation, and if you want to get access to enterprise repository, the minimal community subscription is a no brainer. Ecosystem and third party apps around KVM do not compare to VMware’s of course but you do get some like Acronis backup if the built-in system does not meet your requirements.

Conclusion

Hopefully it is now much easier to pick one. Remember that you can always nest one hypervisor within another. If you need to learn about one but prefer another for your environment, nesting hypervisors is the answer. Bear in mind that although this is possible on Hyper-V, it needs a lot of work and may not be very or as stable as KVM and ESXi.

I personally would recommend Proxmox and KVM over the other two. It provides a fully featured and open source solution. The underlying OS is Debian Linux and you have the flexibility of a typical Linux system.

At the same time, I would also recommend VMUG Advantage subscription, because many if not most enterprise environments utilize vSphere and vCentre and it’s a must-have skill set.

For me, it’s goodbye ESXi & Hyper-V and Hello Proxmox/KVM

Just before new year, I decided to migrate my home lab from ESXi and Hyper-V to a cluster of Proxmox servers. I used CloneZilla to backup VMs from ESXi and Hyper-V, then re-create and restore on Proxmox. This worked very well and without any issues, even with UEFI boot and a database server. I’ve successfully tested live migration between the two servers and being able to assign a generic KVM CPU makes that possible and reliable.

The move to Proxmox was triggered by some annoyances on vSphere:

  • I could not use vMotion (I had tried before installing Hyper-V on Celeron server). Even with EVC or per-VM EVC it was at best very unstable and unreliable. I can understand as the Celeron server is not even on VMware HCL and there are major differences in CPU instruction sets between Celeron and Xeon-D. But KVM gets around this by providing a generic CPU.
  • No vMotion meant I couldn’t move vCentre to another host for ESXi updates and had to do that manually. It’s not a show stopper but becomes annoying. With Proxmox, management can be done from any hosts in the cluster, which is definitely a better implementation.
  • vCentre appliance taking up 10GB of memory was no fun either. In a small environment this is really overkill and takes away from your projects.

So does that mean I won’t ever use ESXi or Hyper-V? No. As a matter of fact I still have the VMUG Advantage subscription and nest both hypervisor on KVM to test and learn.