I wonder what motivates apple to release features like RDMA which are purely useful for server clusters, while ignoring basic qol stuff like remote management or rack mount hardware. It’s difficult to see it as a cohesive strategy.
Makes one wonder what apple uses for their own servers. I guess maybe they have some internal M-series server product they just haven’t bothered to release to the public, and features like this are downstream of that?
> I guess maybe they have some internal M-series server product they just haven’t bothered to release to the public, and features like this are downstream of that?
Or do they have some real server-grade product coming down the line, and are releasing this ahead of it so that 3rd party software supports it on launch day?
I worked on some of the internal server hardware. Yes they do have their own boards. Apple used to be all-in on Linux, but the newer chips are far and away more power-efficient, and power is one of the (if not the) major cost of outfitting a datacenter, at least over time.
These machines are very much internal - you can cram a lot of M-series (to use the public nomenclature) chips onto a rack-sized PCB. I was never under the impression they were destined for anything other than Apple datacenters though...
As I mentioned above, it seems to me there's a couple of feature that appeared on the customer-facing designs that were inspired by what the datacenter people wanted on their own PCB boards.
Apple's OS builds are a lot more flexible than most people give them credit for. That's why essentially the same OS scales from a watch to a Mac Pro. You can mix and match the ingredients of the OS for a given device pretty much at will, as long as the dependencies are satisfied. And since you own the OS, dependencies are often configurable.
That they sell to the public? No way. They’ve clearly given up on server stuff and it makes sense for them.
That they use INTERNALLY for their servers? I could certainly see this being useful for that.
Mostly I think this is just to get money from the AI boom. They already had TB5, it’s not like this was costing them additional hardware. Just some time that probably paid off on their internal model training anyway.
And if the rumors are right -- that hardware SVP John Ternus is next in line for CEO -- I could see a world where the company doubles-down on their specialized hardware vs. services.
They’ve done a dip-in-a-toe thing many times, then gave up.
If I was in charge of a business, and I’m an Apple fan, I wouldn’t touch them. I’d have no faith they’re in it for the long term. I think that would be a common view.
The Mac Studio, in some ways, is in a class of its own for LLM inference. I think this is Apple leaning into that. They didn't add RDMA for general server clustering usefulness. They added it so you can put 4 Studios together in an LLM inferencing cluster exactly as demonstrated in the article.
I honestly forgot they still made the Mac Pro. Amazing that they have these ready to ship on their website. But at a 50% premium over similar but faster Mac Studio models, what is the point? You can't usefully put GPUs in them as far as I know. You'd have to have a different PCIe need to make it make sense.
The M2 Ultra has 32 off-world PCIe lanes, 8 of which are obligated to the SSDs. That leaves only 24 lanes for the 7 slots. That's 8 times less than you'd get from an EPYC, which is the kind of thing a normal user would put in a rack if they did not need to use macos.
The annoying thing is there's no ability to control power (or see system metrics) outside the chassis. With servers and desktop PCs, you can usually tap into power pins and such.
AWS is just used for storage, because it's cheaper than Apple maintaining it, itself. Apple do have storage-datacenter at their campus at least (I've walked around one, it's many many racks of SSD's) but almost all the public stuff is on AWS (wrapped up in encryption) AFAIK.
Apple datacenters are mainly compute, other than the storage you need to run the compute efficiently.
I assume a company like Apple either has custom server boards with tons of unified memory on M series with all the i/o they could want (that are ugly and thus not productized) or just use standard expensive nvidia stuff like everyone else.
It’s quite interesting how „boring“ (traditionally enterprise?) their backend looks on the occasional peeks you get publicly. So much Apache stuff & XML.
Makes one wonder what apple uses for their own servers. I guess maybe they have some internal M-series server product they just haven’t bothered to release to the public, and features like this are downstream of that?