I've used both, though not for anything especially complicated. Your guess about the polish level is correct; ESXi blows Proxmox out of the water interface-wise, especially for some specific tasks like configuring networks or managing PCI passthrough devices. As far as "more limited" goes, I can't say I noticed much of a feature discrepancy, although being open and just sitting atop KVM, Proxmox doesn't have any of the artificial limitations imposed by VMware (like a cap on number of vCPUs in the free version, eugh).
One pretty big one, if you're looking at homelab use, is that Proxmox is just Debian, so it's waaaay more likely to run on crap-tier commodity hardware, whereas VMware has all the usual enterprise-level support commitments and so you're very much on your own if you try to force it to run on hardware that isn't on their official compatibility list.
> One pretty big one, if you're looking at homelab use, is that Proxmox is just Debian, so it's waaaay more likely to run on crap-tier commodity hardware
That was my first (and last) hurdle when wanting to try out ESXi...
I booted up the installer only to be greeted by a "not enough RAM" message.
I believe it's 4GB minimum, which my tiny machine had, but ESXi read it as 3.9GB instead of 4.0GB and refused to budge.
Very happy with Proxmox on the other hand! Having access to the underlying Debian is also quite nice if you need to do something that's not supported by the interface.
Excellent point on hardware support. I'm running several years old Xeon chips in my server, and ESXI is already complaining they might not be supported in a future release.
One pretty big one, if you're looking at homelab use, is that Proxmox is just Debian, so it's waaaay more likely to run on crap-tier commodity hardware, whereas VMware has all the usual enterprise-level support commitments and so you're very much on your own if you try to force it to run on hardware that isn't on their official compatibility list.