Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
25GBit/S on macOS and iOS (kittenlabs.de)
63 points by todsacerdoti on May 21, 2024 | hide | past | favorite | 34 comments


I’ve upgraded to a mostly 10Gbit network at an unreasonable expense and think it’s wild that we consider 25G to be “extreme”.

Personal computers came standard with 1Gb interfaces in the early 00’s. What other I/O standard has made zero progress in 20 years?


2.5G is pretty common on new hardware, 10G is relatively common on high-end prosumer hardware. So not exactly zero progress.

I think a lot of it is down to the realities that most people are only really interacting with the internet at home, and for the most part, speeds over 1gb outside data centers was pretty rare. So very little need to upgrade.

No mass movement to upgrade, means fewer component sales, lower production means economies of scale don't apply. Meaning you're stuck in relatively early adopter territory for a while.

I upgraded my home lan as well last year, my main switch is now 10gb copper, my desktop and nas are the only devices with 10g nics, a few others are 2.5, and my secondary switches are all 2.5g to save money.


2.5G nics are 10gbit capable nbaseT transceivers that ship with 5gb and 10gb rates disabled to create lower end skus. It's a sick joke. The standard should have not permitted disabling the faster rates.

Your average consumer cant really do much with networked data at home because the vendors are not shipping network interfaces that are useful for anything. I can have a laptop with 10/20/40gb USB ports but they can only be bothered to put a 1gbe port on it? Get real.


>Personal computers came standard with 1Gb interfaces in the early 00’s.

Standard? Definitely not. Only high end prosumer gear came with gigabit as standard in the early 00's. The vast majority of affordable consumer gear still had 10/100 Fast Ethernet well into late 00's and I even saw new laptops with it even early 2010's(!).


Is that really a zero progress? It's not insane, but not a zero. You just said you are on 10Gbit now, right? :)

1G has wide adoption, runs on any cheap 5e cable and has low power draw. 10G may or may not run on same cables and has 4-7x the power consumption. That means more heat, more bills, stiffer cables that are harder to manage and so on. I dropped 10G/2.5G attempts at home after realizing that it will mean way to much heat in a TV cabinet somewhere.

A lot of devices don't need anything faster either. If you run, for instance, monitoring setup with POE cameras, you may even use 100Mbit as it's sufficient AND some end devices will have just that (and to my surprise there are still plenty of 100Mbit devices being sold). A lot of people don't realize how many TV's still use 100Mbit. Mine does, yet it can play any 4k video I found. And it's still way above what is actually needed. A 20GB/hour video file is way to big to be reliably played with a lot of capable but slightly older smart devices, tv boxes and so on, and that is HALF of what 100Mbit can provide.

Also - wireless. It's not exactly obvious to have decent wifi coverage at a home. Often it requires effort to be done right. So even when on 25G it won't matter that much if perceived performance suffers because of laptop <-> wireless access point setup.

Then there is storage - getting 50-100MB/s performance is achievable using one cheap 3.5" drive. But if you need some bigger storage, redundancy, or better performance, you need a fairly large investment. It's not always needed and not always justified.

2.5G is showing up more and more, it's not a huge jump, but unlike 10G, it does show up in consumer devices randomly - SFF PCs, stock ISP routers. And it seems to have much less power draw as well.

So 10G is present where it was most desired - for trunk connections to ISP/between major elements like main switch and NAS; where people work on large volumes of video and so on. And obvious in commercial/enterprise setups. But it's not worth going "let's slap 25G on all ports for this consumer device".


10GBase-T has high power draw because the original ICs that implemented it were old; modern nbaset transceivers consume much less power at 10gbit than they used to. The problem is mitigated to a great degree by economies of scale and being able to make new transciever ICs with new process nodes has good benefit. Unfortunately manufactures seem to be content to stall at the existing 2.5gbit skus, so there is not much incentive to produce power efficient 10gbase-t products; those markets are already served by SFP+ direct attach.

For applications where power is actually a concern and bandwidth needs are not high, you can always simply run a lower rate. It's not necessary to completely kill the higher rates in hardware.



Those Mellanox cards are great and surprisingly inexpensive. I just put 4 of the dual QSFP+ cards in servers in my homelab and now have 80gb/s from each server. The Mellanox and Arista QSFP+ switches are also super cheap, and you can get QSFP+ DAC cables to do 40G.

In no way is 25G or 40G extreme. It is commodity low end in the real datacenter market.


How noisy are the Mellanox and Arista switches you're using?

Asking because I'm looking for something suitable for a homelab (using Mellanox 40GbE cards) that's in the same room I tend to occupy during the day.

When I used very early Mellanox switches (years ago), they were only suitable only for datacenters. aka permanent super noisy jet engine mode

Their 1U 10Gb/s Infiniband switch is still the noisiest piece of equipment I ever made the mistake of buying! (!!!)


Indeed - They are loud. ;) You can slow the fan speed down some and that makes a difference, but they still produce a lot of fan noise:

If you listen in this video of my homelab: https://www.youtube.com/watch?v=-b3t37SIyBs

I would say that more than half of the total 'volume' is attributed to the three Arista switches. They are quite a bit louder than the Ciscos, and of course much louder than the Ubiquiti.

I certainly would not have one of these next to me in a work area.

On the other hand, they are the cheapest way to get 40G! With DAC cables it can be really inexpensive.


Thanks, that rules out the Arista option then. :)

I'm currently looking into the HP 5900AF-48XG-4QSFP+ (JC772A) switches, as they seem to be a good price/performance mix on Ebay. :)

My homelab is currently just a small setup with 4 boxes using 40GbE and a few others with 10GbE. Only one of them really needs the 40GbE though, the other's could be made 10GbE and not care. ;)

Random example of this switch:

https://www.networktigers.com/products/jc772a-hpe-5900af-48x...

Can't seem to find any info about noise levels though. :(


Found a data sheet with the acoustic info (https://s4u.com.ua/assets/files/hp%205900.pdf). 65.7-70.6 dB.

That's clearly not going to be workable. Damn.


Yea, a lot of them are pretty high flow due to the density of heat generated. Most are consuming 100-200watts, and that has to all come out as heat!


So - you could consider the Mikrotik - CRS326-24S+2Q+RM

It has 2 40G, and it is very quiet (I have one)


Thanks, that seems like the right kind of thing. :)


Nifty homelab by the way. :)


The arista switch's have an option to limit their max fan speeds, so they go from data centre deafening to as loud as a 1u server.


Yea, it is impressive when they first turn on. You really start to think there is an aircraft overhead!


> as loud as a 1u server.

Depending on the 1u server, that's still super noisy. :(


Exactly lol.


I wonder if this ability is used anywhere professionally, specifically DPX (professional non-video compressed "codec") video color grading with Davinci Resolve. It doesn't make sense for Apple to include drivers otherwise, other than not bothering to shrink down the base install. That or some custom apps via TestFlight for other high bandwidth use cases.


I'm wondering if it may actually be more related to the rumored M* Series servers they've been building for internal use. Would make sense they would want the connectivity / backbone for their AI farm.


The cards seem like they'd physically plug in to the 2023 Mac Pro, without the extra external chassis setup.

So, maybe Apple added those drivers for people using ConnectX adapters with Mac Pros?


This comment is an offtopic rant about the bane of 2.5gbit nics. A discussion about how awesome and cheap 25gbit nics are (on iOS devices no less!) seems as good a place as any to bring it up.

2.5gbit nics are a boondoggle. They must contain the same 10gbit capable transceivers as any other n-base-t transceiver (because that's how they work) yet they are sold with the 5gbit and 10gbit rates disabled. Ostensably this is to limit power draw (and heat) but in most applications they are used those things aren't actually much of an issue.

What will it take to get non-nerfed 10gbit nics as standard offerings on consumer hardware? 2.5Gbit is simply not fast enough to move data in and out of modern systems with modern storage. It's a twisted joke that network connectivity is so shit on systems that otherwise bend over backwards to advertise how many thunderbolt and 10gbit usb ports they have.


> to limit power draw (and heat)

Yes. Did you ever been in an actual DC and handled 10G-T cables and ports?

They are hot, even in the DC with HVAC running below 20°C.

And there is only one type of the systems which can easily accept that amount amount of heat - the desktop *ATX systems, for which you can just go and buy a 10G PCI-E card.

For everything else the amount of heat is too much.


So yeah I do deal with 10GBaseT sometimes. IMO its mostly an economies-of-scale issue. 802.3an is up against some physical limits, but there have been many improvements even since the first products hit the market. The newer PHYs used in the newest multirate SFP+/BaseT transceivers, for instance are not nearly as hot as the old 1st gen products. More users = more incentive to fix the problem.

I wonder if it might be possible via software/firmware hack to unlock the 5G/10G rates on any existing 2.5G chipset?


DACs don't seem to have that problem.


DAC definitely has that problem too. One cable in an ideal condition probably wouldn't be that hot, but as soon as something is not ideal it would get hot.

Also you are just throwing everything here now, DAC is totally not "we can have 10G for free". As others has said, the main benefit of 1G/2.5G is what it uses the same twisted pair => 8p8c connector. 10G DAC is SFP+. This alone prevents the mass adoption in the consumer sector - too much friction for Regular Joe.


> Also you are just throwing everything here now ...

?


The advantage to 2.5Gb/s is that you can run it over reasonable lengths of existing crappy cable jobs about as well as 1Gb/s Ethernet without angry customers yelling blaming you because it doesn't work. sigh


I had always wanted to port Mellanox drivers to macOS to get away from paying through the nose for ATTO cards.

Great news that Apple have already done it!

Mellanox ConnectX 6 Lx is pretty cheap in comparison to ATTO.


There are reasonably priced ConnectX-4's on Ebay these days too. :)

Random example (I don't know the seller): https://www.ebay.com/itm/166609931874

That's $39.99 for a dual port 25GbE card, and that generation are still on the official supported list for lots of things.


I just want to plug an nVidia GPU into my macbook pro.


Good luck with drivers!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: