I was at embedded world this week, and Asrock, Gigabytes, MSI were all showing rack mounted servers with consumer grade Ryzens.
Just by looking at revision numbers at their boards, it seems clear that they already went through quite a number of iterations, just as usually happens with more proper server mobos.
Historically I've always gone with Intel, but I've been wondering if new AMD CPUs would work well hypervisor hosts. A Ryzen Threadripper 3990x for example has 64 cores, 128 threads, and supports ECC RAM. Intel by comparison doesn't offer anything close to those numbers on a single CPU. So, does anyone have any thoughts as to why using the 3990x or some other similar AMD product wouldn't be a great idea compared to using some Intel solution? Is the, say, dual Xeon approach better in some way I don't know? I can't tell if Intel is actually better these days or if they just benefit from all the contracts they've been able to get with major manufacturers over time.
Something to keep in mind if you use it, VMware just changed their licensing for CPUs over 32 cores require an additional ESXi license. So that 64 core processor now requires 2 licenses instead of one!
I can tell you why I just choose a dual Xeon workstation over a thread ripper, despite it being considerably more expensive for lower performance: memory. The threadripper motherboards apparently max out at 256 GB of ram. I put 1 TB on the Xeon machine and could have gone higher if I needed it.
It is really too bad, but 64 cores with a max of 256 GB is super unbalanced.
I clicked that link expecting Linus Torvalds and was very confused. The mononym "Linus" is already taken :-) It would be very exciting to see Linus Torvalds playing with giant amounts of RAM but sadly he doesn't seem to make a lot of videos like that.
If you're talking operating systems, it's Linus Torvalds. If you're talking tech videos, it's Linus Sebastian. Linus Media Group gets a lot of the cool hardware first and often makes a point of getting some unofficially to mess with it in ways they're not supposed to.
Ryzen 3rd gen max memory is 128GB, which matches a realistic requirement for doing a cookie cutter cloud hosting.
A wicked fast quad core instance with 32 gigs of mem is a top tier offering as far as cloud hosting goes.
And as for cost/perf goes, Ryzen U1s can easily yield more money than a mid-tier Xeon offering, especially if you are buying twins or quad systems (2 or 4 independent systems in a single U1 enclosure)
Definitely an important factor, didn't check the max RAM. Kind of surprises me that AMD would put all this work into building a many core/thread CPU only to have a 256 GB RAM limit but if it's intended for gamers/workstations and not servers then I guess 256 GB would be plenty for that. But I agree it's unbalanced regardless.
That's the market -- EPYC is the server oriented product, and it supports a lot more RAM, though I think still less than Intel (I hand-wavily remember 4TB for EPYC, 6TB for Xeon, but with a huge price difference between them).
Given the design of threadripper, it's not really "all this work" to make a 64 core model, given the 32 core model. Just pile more of the things you can scale on the chip without worrying too much about the parts that don't scale. And the same happened with previous generation. 16 core 2950X turned into 2990WX, but with a weird crippled memory interface. At least 3990X is more balanced than that.
I was looking for adding at least one Threadripper server to our local HPC system. I only found one ASRock motherboard for TR4, but none yet for sTRX4. I hope we will see 1U rackmounted sTRX4 systems soon.
Our new HPC system will mostly consist of Epyc 7742, but having one node with super high single-thread performance would be nice for less well parallelized applications.
> I can’t see enterprise vendors selling them and I doubt the support is great.
Hosting companies don't end with enterprise cloud stuff. The availability of consumer grade hardware chips has always been much better than server ones for anybody, but tier 1 vendors.
Just before there was little incentive to chase that market for anything as even the best consumer grade CPUs still were not contenders on core count, ECC, IPMI support, and cache size to compete with Xeons.
But now, even very cheap 3rd gen Ryzens can easily beat mid-tier Xeons: more cores, more cache, ECC support, support for Aspeed remote management. And they are way more power efficient, so making a twin or even a quad system in U1 is possible.
MSI guy was particularly secretive. MSI appears to actually have a separate sub-brand they don't market around to do server OEM AMD products without loosing Intel partner status.
There is definitely. Particularly for some applications requiring the highest possible frequency (Ryzens are available in higher frequencies than Epycs). Plus Ryzens support ECC so there are fine server CPUs.
Just a note, as I was recently slightly disappointed when shopping for RAM for a recent Threadripper build:
The current max capacity for Threadripper ECC RAM is 128GB. The largest unbuffered ECC DIMM is 16GB. With an Epyc (or a Xeon) you could used registered memory which is available in higher capacities.
If you went 3990X, you'd be limited to 1GB per thread or 2GB per core. This isn't a showstopper, but could be a bottleneck depending on how RAM-intensive your workload is.
Let me be clear, I'm still quite happy with my recent purchase and build. Threadripper has enabled unprecedented core counts at very low prices.
Along with that, I've not reliably been able to find fast unbuffered ECC ram either. I've seen some 2333MHz stuff but not in 16GB DIMMs and nothing faster actually for sale (I've seen 2400MHz advertised but not actually available). I've got 128GB of 2133MHz and it works fine but it'd be nice to be able to get the extra performance considering how much it matters for the Zen core cpus
Crucial has 16GB unbuffered ECC at 2666MHz native frequency, available within the last month. It's a very stable overclock at 3000MHz. I think I've got room to move up to 3200. I would love to get to 3600MHz, but that has seemed unstable when I've tried.
CAS latency isn't the best, and tweaking this has resulted in a quite unstable system.
Neat, that's definitely new and good to hear about. I'm eventually going to upgrade my 1950x and the rest of the system so getting to 3000 would make a huge difference already. I've managed to get my sticks to 2333 but they then report about 3 errors a week, and about 20 a day at 2666. The performance gains from that aren't enough to leave it like that and feel good about it to me.
I can post a link to a full PCPartPicker list later if you want, or answer specific questions.
It's a 3970X with 128GB (link to RAM in sibling thread) of 3000MHz RAM on a GIGABYTE Aorus Master. No processor overclock. Running Noctua's NH-U14S with one of their high-RPM fans. GPU is nothing special - the cheapest card I could find with DisplayPort 1.4+ and HDMI 2.0+ (for potential dual 4k@60Hz - currently dual 2560x1440@60Hz) - it's a Radeon 550. Boots from NVME and has 40TB raw (20TB usable) spinning disk.
It's a virtualization host and prototyping workstation. I work primarily in data analytics, and it's not uncommon to have prod servers larger than this. I've got enough headroom to realistically test things at scale.
Just by looking at revision numbers at their boards, it seems clear that they already went through quite a number of iterations, just as usually happens with more proper server mobos.
There must be a demand for them.