> For some more exotic testing, I was able to run my own magic wormhole relay[1], which let me tweak some things for faster/more reliable huge file copies.
The lack of improvement in these tools is pretty devastating. There was a flurry of activity around PAKEs like 6 years ago now, but we're still missing:
* reliable hole punching so you don't need a slow relay server
* multiple simultaneous TCP streams (or a carefully designed UDP protocol) to get large amounts of data through long fat pipes quickly
Last time I tried using a Wormhole to transmit a large amount of data, I was limited to 20 MB/sec thanks to the bandwidth-delay product. I ended up using plain old http, with aria2c and multiple streams I maxed out a 1 Gbps line.
IMO there's no reason why PAKE tools shouldn't have completely displaced over-complicated stuff like Globus (proprietary) for long distance transfer of huge data, but here we are stuck in the past.
I overall agree, but "reliable holpunching" is an oxymoron. Hole punching is by definition an exploit of undefined behavior, and I don't see the specs getting updated to support it. UPnP IGD was supposed to be that, but well...
Well, with v6 you're down from NAT-hole-punching to Firewall-hole-punching, which in principle should be as simple as arranging the IP:Port pairs of both ends via the setup channel, and then sending a "SYN" packet in both directions at once.
Then, trying to use e.g. TCP Prague (or, I guess, it's congestion control with UDP-native QUIC) as a scalable congestion controller, to take care of the throughout restrictions caused by high bandwidth delay product.
That's why protocols like this have what Wormhole calls a "mailbox server", which allows two ends separated by firewalls to do secure key exchange and agree upon a method for punching through directly. See also STUN: https://en.wikipedia.org/wiki/STUN
I'm working on a branch that considerably improves the current code and hole punching in it works like a swiss watch. If you're interested you should check out some of the features that work well already.
I (Electrical + Software Engineer) once worked for a physicist who believed that anything less than an order of magnitude was merely an engineering problem. He was usually correct.
I was taught the same. To not care a lot about things under an order of magnitude. Over the years when planning large software projects or assessing incidents and so on, the 1 order of magnitude threshold helped me often.
As a physicist, I think this is correct too :). You don't start to see problems with things under that, unless they are deviation from standard model predictions.
Not as far off as the casual reader might think 20MB vs 1Gb sounds way more than the actuall 160Mb vs 1Gb - one shouldn't use Bytes and bits in a direct comparison together. One or the other, otherwise it's misleading/confusing.
In this case transferring the data at the slow rate would have taken more than a week, so it's no small difference. Actually one side had a 10 Gbps line, so if the other side had had faster networking I could easily have exceeded the limit and gotten the transfer done more than 6x faster.
I used the term "1 Gbps line" just because it's a well known quantity - the limitation of Gigabit Ethernet. The point wasn't that multiplexing TCP can get you 6x better speeds, it's that it improved the speed so much that the TCP bandwidth-delay product was no longer the limiting factor in the transfer.
Yeah but with magic wormholes you see, there could be other universes where that's not the case and 160mbps is close to 1024mbps or 1000mbps whatever the cool kids call a gigabit now adays.
As a protocol tcp should be able to utilize a long fat pipe with a large enough receive window. You might want to check what window scaling factor is used and look for a tunable. I accept that some implementations may have limits beyond the protocol level. And even low levels of packet loss can severely affect throughput of a single stream.
A bigger reason you want multiple streams is because most network providers use a stream identifier like the 5-tuple hash to spread traffic, and support single-stream bandwidth much lower than whatever aggregate they may advertise.
Yeah, that's the issue. I didn't have root permissions on either side. Moreover, a transfer tool should just work without requiring its users to have expert knowledge like this.
In this case, I checked the roundtrip ping time and multiplied it by the buffer size, and it agreed with the speeds I was seeing within ~5%, so it was not an issue with throttling. Actually, if I were a network provider interested in doing this, I would throttle on the 2-tuple as well.
The lack of improvement in these tools is pretty devastating. There was a flurry of activity around PAKEs like 6 years ago now, but we're still missing:
* reliable hole punching so you don't need a slow relay server
* multiple simultaneous TCP streams (or a carefully designed UDP protocol) to get large amounts of data through long fat pipes quickly
Last time I tried using a Wormhole to transmit a large amount of data, I was limited to 20 MB/sec thanks to the bandwidth-delay product. I ended up using plain old http, with aria2c and multiple streams I maxed out a 1 Gbps line.
IMO there's no reason why PAKE tools shouldn't have completely displaced over-complicated stuff like Globus (proprietary) for long distance transfer of huge data, but here we are stuck in the past.