> But if you want to participate in the writing, debugging, and maintenance, it has to be in a language that a human can read.
I think the idea is that languages like Python and JavaScript make it easier for humans to write the initial implementation, whereas the "hard" languages from the perspective of creating the minimum viable product are the ones that make it easier for humans to maintain the code, and this has historically been a major trade off.
Whereas if you have the AI write the initial implementation...
> You cannot utilize that type of speed with a Mac Mini.
Mostly because the base Mini has Thunderbolt 4 which maxes out at 40Gbps. Anything with a PCIe 4.0 x16 slot will take a 100Gbps NIC. 100Gbps is around 10GBps (8 bits per byte plus encapsulation overhead). Desktop CPUs can do AES-GCM at 2.5GBps+ per core and have up to 16 cores and around 50GBps of memory bandwidth (dual channel DDR4-3200), so the NIC still seems like the bottleneck.
The noise problem is pretty easy to mitigate by choosing 2U servers instead of 1U. The latter are forced by the form factor to use smaller, higher speed fans.
A bigger issue for enterprise hardware is that it's optimized for performance per watt under load, not idle power consumption. Running a mostly-idle rack server 24/7 can result in a pretty sizable electric bill. This also depends heavily on the model. Some will idle at ~50 watts, others at ~300, but both of these are significantly higher than a Raspberry Pi or an old laptop which for personal use will generally do the job.
Business class desktops are also a good alternative here. Many models have pretty reasonable idle power consumption (check this for yourself, I've seen 6W but also 60W) and then you get a couple of drive bays and PCIe slots and expandable RAM which you don't get from a Raspberry Pi.
These days, pretty much the only thing that makes sense is a mini PC. AMD laptop chips generally trade blows with Apple stuff on power efficiency when you thrash them, and you get a surprisingly capable machine for not very much money.
It's really not worth it to run old hardware 24/7 unless it's making money. Buying a new machine of equivalent capability is (normally) pretty cheap, and it doesn't take very long for the power savings to pay for themselves.
They can be had with fairly respectable specs too. Certainly enough to play around with small local models.
"When you thrash them" is kind of the issue. There are ten year old business desktops with a <10W idle power consumption. If your use for it is to have something to rsync files to and host your personal website and the like, even old hardware is going to average 99% idle. There is no meaningful power savings from newer hardware unless you're consistently putting it under significant load.
Some of the newer hardware is actually worse because the idle power consumption of PCs since around 2010 is determined in significant part by the low-load efficiency of the power supply. Brand new machines with the wrong power supply can use several times as much power at idle as ten year old machines with the right power supply. Annoyingly, power supply efficiency at idle is rarely documented so the only thing to do is measure it.
The problem with fibre isn't the sensitivity. It's that most endpoints have a 1Gbps copper port on them and then Cat6A ports can be used with the common devices but also allow you to add or relocate 10Gbps devices without rewiring the building again.
You've been able to get Intel X520 NICs [0], with transceivers included for ~40USD on Newegg for a long time. This is a little more than double the price of Newegg's cheapest single-port 10/100/1000 copper card, but even the cheapest available such card is three times your "chicken and egg"-solving price point.
I suspect the combination of the absence of cheap-o all-in-one AP/router combo boxes with any SFP+ cages and fiber cabling's reputation of being extremely fragile have much more to do with its scarcity at the extremely low end of networking gear than anything else.
But +$15 and an extra wall outlet per endpoint is still an inconvenience, and if a two-port device with its own power supply can be made for $15 then where is the PCIe/USB to fibre adapter for <$10?
Yep. Good NICs last for approximately forever, life's way too short to deal with maybe-flaky NICs, and the price difference between the Amazon Special and something that's going to be reliable is -what- two big boxes of Cheerios? Two dozen eggs? Not. Worth it.
> But it's not competing with those, it's competing with the copper port which is already built into most devices.
Correct! That's part of why I was so very surprised to see you suggesting that extremely cheap PCI Express and USB adapters would "solve the chicken and egg problem".
> The point being you need some cheap way to plug in existing copper devices if you run fibre to the endpoints.
That's called a multi-port switch. Netgear sells five-port gigabit ones for like 20 USD. Switches that have two SFP+ cages and eight copper gigabit ports [0] are six times the price of a cheap-o Netgear switch, but are something that's going to last at least a decade. It's also pretty uncommon to find SOHO switches that have SFP+ cages and don't have at least one fixed copper port.
> This plus $5 for a transceiver is pretty close at $15:
If you're connecting a single device, why the hell would you use that when you could slap a copper SFP or SFP+ module in the switch's cage and run a cable? If you're connecting multiple devices, then either install multiple copper modules and run multiple cables, run multiple copper cables from fixed copper ports on the switch, or put a switch where the existing copper devices are.
> If you're connecting a single device, why the hell would you use that when you could slap a copper SFP or SFP+ module in the switch's cage and run a cable?
The problem to be solved is that you want to be able to put fibre inside the walls of the building instead of copper. Running a new cable to the switch closet is the thing to be prevented.
But if the wall jacks are fibre then you need some economical way of hooking them up to every printer and single-purpose device with a network port. If you have to buy another $100+ switch just to get from fibre to copper even when there is only one device near that jack, people aren't going to go for that.
> The problem to be solved is that you want to be able to put fibre inside the walls of the building instead of copper. Running a new cable to the switch closet is the thing to be prevented.
...why would you ever not run copper alongside fiber for new construction? If nothing else, PoE is extremely useful, and nothing says that you actually have to connect all of that copper cable to your switch... you can connect it as-needed. I also can't imagine that most refits only have room for exactly one cable in their conduit. [0]
I'd expect to hear the sort of plan you propose from a PHB or Highly Paid Consultant, not someone who actually has had to use that sort of configuration.
Regardless, the scenario you're now proposing is one where noone other than a PHB would use that Amazon Special that you linked for media conversion.
[0] If there's no conduit and cables are all flopping around in the wall, then there's even more room for cabling.
The original problem was that everyone runs copper instead of fibre because there are too many existing devices that only have copper. Running both everywhere would require you to buy and terminate twice as much cable as you expect to use, which leads people to running only copper again.
If you chose PCs to begin with that come with fibre ethernet or put quality cards in the ones that matter then you could make fibre the default instead of copper. Until you have a number of devices like printers or VoIP phones or Raspberry Pis that have no need for 10Gbps or even 1Gbps connectivity, they just need a way to be plugged in at all. If you need to add $100+ in conversion expense to each of those devices, you're back to using copper by default.
> Running both everywhere would require you to buy and terminate twice as much cable as you expect to use...
Ah. Let's play with that logic a bit:
"Running Ethernet cabling everywhere would require you to buy and terminate far more than twice as much cable as you expect to use. Just run power cables and wire up one extra outlet for a HomePlug in each room.".
Yeah, that checks out. "Powerline Ethernet" devices are actually pretty good these days, and are right around your magic price range... Amazon has them at ~13 USD per unit. [0] Why would anyone bother running a second cable to each room? Thirteen bucks per room has to be way less than the materials and labor cost for the cable run. Doing anything else is, like, really stupid. Don't you agree?
Anyway. You expect to use the cabling that you plan to install... plus some extra for screwups, man.
> You can run composite cable if you desire copper, fiber and power.
Oooh. Cool.
By "power" do you mean 120/240VAC, or do you mean much lower voltage DC? I've found some Belden cabling that I think provides mains power and Ethernet, and I've found fiber cabling that I guess carries lower voltage DC, but am having a tough time finding a cable that combines fiber and copper data with mains power. Do you have an example of such a cable handy?
(Full disclosure: I'm refusing to spend more than like five minutes on the search... so I might have been able to dig up examples of such a cable.)
Which is why people run only copper because that costs less than running multiple types of cable everywhere when most drops only have one device, and then pull fibre through using the existing copper cable in the rare instances where they find a need for 40Gbps or more.
But then the copper gets used for 10Gbps connections instead of fibre because it's what's already in the building.
Ymmv. I've got a mix of cheap premade patch cables and some I crimped from solid core, all cat5e, all holding 10gbe totally happily. I suspect that only works because they're a meter or two long but that reaches across the rack.
> If we can get that to raise a red flag with people (and agents), people won’t be trying to put control instructions alongside user content (without considering safeguards) as much.
At a basic level there is no avoiding this. There is only one network interface in most machines and both the in-band and out-of-band data are getting serialized into it one way or another. See also WiFi preamble injection.
These things are inherently recursive. You can't even really have a single place where all the serialization happens. It's user data in JSON in an HTTP stream in a TLS record in a TCP stream in an IP packet in an ethernet frame. Then it goes into a SQL query which goes into a B-tree node which goes into a filesystem extent which goes into a RAID stripe which goes into a logical block mapped to a physical block etc. All of those have control data in the same stream under the hood.
The actual mistake is leaving people to construct the combined data stream manually rather than programmatically. Manually is concatenating the user data directly into the SQL query, programmatically is parameterized queries.
>All of those have control data in the same stream under the hood.
Not true. For most binary protocols, you have something like <Header> <Length of payload> <Payload>. On magnetic media, sector headers used a special pattern that couldn't be produced by regular data [1] -- and I'm sure SSDs don't interpret file contents as control information either!
There may be some broken protocols, but in most cases this kind of problem only happens when all the data is a stream of text that is simply concatenated together.
The header and length of the payload are control data. It's still being concatenated even if it's binary. A common way to screw that one up is to measure the "length of payload" in two different ways, for example by using the return value of strlen or strnlen when setting the length of the payload but the return value of read(2) or std::string size() when sending/writing it or vice versa. If the data unexpectedly contains an interior NULL, or was expected to be NULL terminated and isn't, strnlen will return a different value than the amount of data read into the send buffer. Then the receiver may interpret user data after the interior NULL as the next header or, when they're reversed, interpret the next header as user data from the first message and user data from the next message as the next header.
Another fun one there is that if you copy data containing an interior NULL to a buffer using snprintf and only check the return value for errors but not an unexpectedly short length, it may have copied less data into the buffer than you expect. At which point sending the entire buffer will be sending uninitialized memory.
Likewise if the user data in a specific context is required to be a specific length, so you hard-code the "length of payload" for those messages without checking that the user data is actually the required length.
This is why it needs to be programmatic. You don't declare a struct with header fields and a payload length and then leave it for the user to fill them in, you make the same function copy N bytes of data into the payload buffer and increment the payload length field by N, and then make the payload buffer and length field both modifiable only via that function, and have the send/write function use the payload length from the header instead of taking it as an argument. Or take the length argument but then error out without writing the data if it doesn't match the one in the header.
>It's user data in JSON in an HTTP stream in a TLS record in a TCP stream in an IP packet in an ethernet frame. Then it goes into a SQL query which goes into a B-tree node which goes into a filesystem extent which goes into a RAID stripe which goes into a logical block mapped to a physical block etc. All of those have control data in the same stream under the hood.
It's true that a lot of code out there has bugs with escape sequences or field lengths, and some protocols may be designed so badly that it may be impossible to avoid such bugs. But what you are suggesting is greatly exaggerated, especially when we get to the lower layers. There is almost certainly no way that writing a "magic" byte sequence to a file will cause the storage device to misinterpret it as control data and change the mapping of logical to physical blocks. They've figured out how to separate this information reliably back when we were using floppy disks.
That the bits which control the block mapping are stored on the same device as a record in an SQL database doesn't mean that both are "the same stream".
> There is almost certainly no way that writing a "magic" byte sequence to a file will cause the storage device to misinterpret it as control data and change the mapping of logical to physical blocks.
Which is also what happens if you use parameterized SQL queries. Or not what happens when one of the lower layers has a bug, like Heartbleed.
There also have been several disk firmware bugs over the years in various models where writing a specific data pattern results in corruption because the drive interprets it as an internal sequence.
I distinctly remember bugs with non-Hayes modems where they would treat `+++ATH0` coming over the wire as a control, leading to BBS messages which could forcibly disconnect the unlucky user who read it.
In this particular case, IIRC Hayes had patented the known approach for detecting this and avoiding the disconnect, so rival modem makers were somewhat powerless to do anything better. I wonder if such a patent would still hold today...
What was patented was the technique of checking for a delay of about a second to separate the command from any data. It still had to be sent from the local side of the connection, so the exploit needed some way to get it echoed back (like ICMP).
DOS had a driver ANSI.SYS for interpreting terminal escape sequences, and it included a non-standard one for redefining keys. So if that driver was installed, 'type'ing a text file could potentially remap any key to something like "format C: <Return> Y <Return>".
You expect the files to still be accessible using relative paths. What do you expect to happen if your cloud storage file path is 50 characters long and is mounted in a folder which is 4050 characters long when PATH_MAX is 4096?
The sync application itself can handle this using openat(2) or similar and should probably be using that regardless to avoid races.
Ah, I forgot that the maximum path length is usually limited by PATH_MAX, it's the path segment that's usually limited by the filesystem.
Point taken, although I still think it's better for cloud storage services to err on the side of compatibility, i.e. what's the lowest common denominator between Linux, macOS, Android, iOS from 10 years ago and Windows 7?
That's not a great idea for three different reasons: Filesystems have to do ugly things when they're almost full like split files into many small blocks and store more metadata to keep track of them all, SSDs get slower and have compromised wear leveling when they're almost full, and it makes you more likely to subject yourself to perils of fully running out which can cause random non-temporary problems even if it only happens temporarily.
A good way to do this is to create a swap file, both because then you can use it as a swap file until you need to delete it and because swap files are required to not be sparse.
If you create the file with 'mkswap --file' it allocates the blocks. Trying to use 'swapon' with an existing sparse file won't remove the holes for you but does notice them and then refuse to use it.
There are different governments and different subdivisions within any given government. The only thing you need to get a government that had been pushing Chat Control to do some trust busting is to get more votes.
I think the idea is that languages like Python and JavaScript make it easier for humans to write the initial implementation, whereas the "hard" languages from the perspective of creating the minimum viable product are the ones that make it easier for humans to maintain the code, and this has historically been a major trade off.
Whereas if you have the AI write the initial implementation...