There are no instances anyone can point to where I've caused regressions in other subsystems, in the course of developing bcachefs. I have had people introduce regressions into my code without following proper channels, though.
Changes should be in place in time for regression testing, if you cannot manage that wait for the next cycle.
Those same rules apply to everyone, and you're being called out for not only repeatedly ignoring them, but also for refusing to listen to that criticism.
LFS has been a tremendous resource over the years for me.
I'm happy it exists and that people put in the time to keep it going.
I often wonder why the existence of long "you also need to do this completely unintuitive thing first" documentation on the open internet isn't shaming more projects into reducing barriers to build their software.
I'm sick and tired of the "the evil attacker attacked this harmless company" rhetoric.
Take some responsibility for your actions!
I loathe that you can apparently get away by telling reporters that it must have been a nation state actor. Oh it was just one kid in a hotel room? Well then he must have autism! Hey have you seen Rain Man? Yeah, must have been that kind of super power autism!
It's revolting. Get your act together and stop blaming kids. If a kid can unlock your door by entering the Konami code on your door bell, that's on you.
Locks are a social contract. We all know they can be bypassed, but doing so is illegal. Kids and adults should be held accountable for repeatedly breaking the law.
I think it's fair to comment not only on the subject, but on the writing itself, too.
And it might help Justine improve her writing (and reach a larger audience -- after all, blog posts intend to reach some audience, don't they?). Of course you can always say, "if you find yourself alienated, it's your loss".
Ladies and Gentlemen, let the race (to the bottom) begin!
While the vultures will shit out AI generated garbage in volume to make ever diminishing returns while externalizing hosting cost to Youtube and co, actual creators will starve because nobody will see their content among the AI generated shit tsunami.
Finally the AI bros are finishing the enshittification job their surveillance advertising comrades couldn't. Destroy ALL the internet! Burn all human culture! Force feed blipverts to children for all I care, as long as I make bank!
I guess it's easiest to destroy culture if you didn't have any to begin with.
This smells like they are using shared memory, which is almost certainly a security nightmare. The way they are selling it makes me fear they aren't aware of what a time bomb they are sitting on.
Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith), or you do hardcore capturing (make a copy of each message in the framework before handing it off). Their web page mentions zero copy, so it's probably not the second one.
Also, benchmarks are misleading.
It's easy to get good latency if your throughput is so high that you can do polling or spin locks, like for example in benchmarks. But that's probably not a good assumption for general usage because it will be very inefficient and waste power and require more cooling as well.
> Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith)
There are all sorts of domains where mutually-trusted parties need IPC. Off the top of my head and in no particular order:
- Applications that pass validated data to/from captive subprocesses. Not everything is available as a natively-linked library. Not every language's natively-linked libraries are as convenient to reliably install as external binaries.
- Parallelism/server systems farming work out to forked (but not exec'd) subprocesses. Not everything needs setuid. Somtimes you just want to parallelize number crunching without the headache of threads (or are on a platform like Python which limits threads' usefulness).
- Replatforming/language transitions in data-intensive applications. Running the new runtime/platform in the same address space as the legacy platform can bring some hairy complexity, which is sidestepped (especially given the temporary-ness of the transitional state) with careful use of shared memory.
And aren't systems like Postgres counterpoints to your claim? My memory isn't the greatest, but IIRC postgres's server-side connections are subprocesses which interact with the postmaster via shared memory, no?
If you use shared memory with a captive process, that process can probably hack you if it gets taken over by an attacker.
I agree with your parallelism counter-argument in principle. However even there it would probably make sense to not trust each other, to limit the blast radius of successful attacks.
In your next point the "careful" illustrates exactly my point. Using shared memory for IPC is like using C or C++ and saying "well I'll be careful then". It can work but it will be very dangerous and most likely there will be security issues. You are much better off not doing it.
Postgres is a beautiful argument in that respect. Yes you can write a database in C or C++ and have it use shared memory. It's just not recommended because you need professionals of the caliber of the Postgres people to pull it off. I understand many organizations think they have those. I don't think they actually do though.
> Using shared memory for IPC is like using C or C++ and saying "well I'll be careful then"
Sort of. In the replatforming/codebase conversion case, the amount of care and labor needed to effectively use shared memory for communication is, in my experience, much less than the care and labor needed to make things work in the same address space (and the labor and cost-in-currency of making things work with sufficient performance using traditional IPC is also often preventative). In that regard, I don't think it's equivalent to arguing in favor of C: capable alternatives to C exist that require less care and labor to use; capable alternatives to shared memory IPC in some cases do not.
I'd be curious how much effort and specific shared-memory-IPC-expertise is required on the part of the Postgres maintainers to keep the shared memory layer capable and secure. I hope it's more like iceoryx2 bills itself in that it's relatively well encapsulated such that folks can use it safely without extensive familiarity with the risk domain, but I might be disappointed.
Regardless, I hope we can agree to disagree on specifics; I appreciate the thoughtful response in any case.
> I'd be curious how much effort and specific shared-memory-IPC-expertise is required on the part of the Postgres maintainers to keep the shared memory layer capable and secure.
I don't think security plays a huge role for shared memory in postgres - if an attacker gains arbitrary code execution in one postgres backend, the installation is hosed. No need to go through SHM to escalate to other backends, there's easier ways.
WRT capable: There's definitely substantial costs due to using inter-process shared memory. But it's more an architectural cost, rather than something that everyone has to bear while just doing mostly unrelated hacking. Some features get harder, less flexible and require more code.
FWIW, there's some work towards moving towards a threaded connection model. Still some way to go, but I expect it to happen eventually.
Shared memory would be two processes that can already do whatever they want communicating with each other. What is it that you think is a 'security nightmare' ?
what a time bomb they are sitting on
You didn't give any real evidence of this or examples.
Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place?
Because you can have two or more different processes communicate asynchronously. They are in their own memory space and running on different threads. One doesn't crash the other. All they need to work together is data structures and data formats.
Don't forget that files are the original IPC.
Also, benchmarks are misleading.
Saying something is wrong is easy when you don't have anything to show that it's wrong.
that's probably not a good assumption for general usage
Then don't do it. Shared memory can use atomics, it can be totally lock free. You each process do checks that are just atomically reading and integer.
> Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith), or you do hardcore capturing (make a copy of each message in the framework before handing it off). Their web page mentions zero copy, so it's probably not the second one.
This is an extremely puzzling comment. I can think of thousands of such cases.
First, there are many reasons to split your program into processes instead of threads (e.g. look at browsers) so even if you have a monolith, you may need IPC between trusted parties simply because of software engineering practices. As a more extreme example, if you're writing code in a language like Python, where multi-threading is a huge liability due to GIL and the standard solution is to just use multi-processing, you'll need a channel between your processes (even if they're just fork()'d) and so you need to use something like filesystem, unix pipe, postgresql, redis, some ipc lib (e.g. TFA)... whatever as a way to communicate.
Second, your comment implies there is no scenario where implementing two separate programs is preferable to building a monolith. Even though you believe in general monoliths are better, it doesn't follow that they have to always be the right approach for every software. You may have a program that requires extremely different computational techniques, e.g. one part written in Prolog because it needs logical constraint satisfaction solving, or one part needs X language because you have to use a specialized library only available in language X, or you may need one part of your program to be in C/C++/Go/Rust for improved latency, or you may need part of your program in "slow" Y language because every other codebase in your company is written in Y. This language barrier is simply one reason. I can come up with many others. For example, parts of the software may be developed by two separate teams and the IPC is decided as the interface between them.
Long story short, it's pretty normal to have a monolithic codebase, but N processes running at the same. In such cases since all N processes are written by you, running in hopefully-trusted hardware, using an IPC framework like this is a good idea. This is not necessarily the most common problem in software engineering, but if you do enough systems programming you'll see that a need for IPC between trusted processes is hardly niche. I personally reach for tools like iceoryx quite frequently.
> This smells like they are using shared memory, which is almost certainly a security nightmare.
Yes, we are using shared memory, and I agree that shared memory is a challenge but there are some mechanisms that can make it secure.
The main problem with shared memory is, that one process can corrupt the data structure while another process is consuming it. Even verifying the contents of the data structure is insufficient since it can always be corrupted afterwards. We have named the problem "modify-after-delivery problem" - a sender modifies the data after it has been delivered to a receiver.
This can be handled with:
1. memfd: The sender acquires it, writes its payload, seals it so that it is read-only and then transfers the file descriptor to all receivers. The receiver can verify the read-only seal with fcntl. Since linux guarantees us that it cannot be reverted the receiver can now safely consume the data. This allows it to be used even in a zero-trust environment. [1] provides a good introduction (see the File-Sealing IPC subsection).
2. Memory protection keys [2]: I do not have too much experience with them, but as far as I understand, they solve the problem with mprotect, meaning, the sender can call mprotect and make the segment read only for it self, but the receiver has no way of verifying it or to prevent the sender from calling mprotect again and granting it read/write access again to corrupt the data.
So, the approach is that a sender acquires shared memory, writes its payload into it, makes it read-only, and then transfers it to the receivers.
> Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place?
Robustness is another use case. In mission-critical systems you trust each process but a crash caused by a bug in one sub-system shall not bring down the whole system. So you split up the monolith in many processes and the overall system survives if one process goes down or deadlocks, assuming you have a shared memory library that itself is safe. If you detect a process crash, you can restart it and continue operations.
Holy shit these claims are wild!
It's not just a percent more performance here and there, the graphs look more like 50% more throughput on the same hardware (depending on the cpu architecture).
My immediate fear was that they optimized away the security features like absence of timing side channels, but they say they still have those.
They also claim to have formal proof of correctness, which is even more amazing, because they are not doing it on a symbolic level but on a machine instruction level. Apparently they tought their reasoning system the semantics of all the CPU instructions used in the assembler implementation.
I'll still wait what djb has to say about this, but it looks freaking amazing to me.
> After two sets of disastrous quarterly earnings the company’s market value has shrivelled to $84bn, less than the value of its plants and equipment, from over $210bn in January.
Holy smokes. I just looked at the INTC 5 year chart and it is, in fact, a bath of blood. I don't think it happens very often that the market cap of a company is less than the worth of its physical assets.
There was a time when Apple had a market cap approximately equal to its book value, about 4 billion.
This was when basically every article about the company used the word beleaguered, they couldn’t actually ship the computers people wanted to buy, and they were flailing around with their next gen os.
It's hard to value said assets. You'd need a buyer for the fabs to run them as a going concern for them to be worth more than the scrap value and it's not obvious who that buyer might be. Piles of stock noone wants to buy probably aren't assets either, and the expensive office space might not be either.
Being worth less than your assets and liabilities is an extremely dangerous position to be in. The only reason they haven’t been bought and stripped for parts is the fact that the government probably would block it.
I was contemplating the same thing. I get Intel isn't doing so well but I really can't imagine it just collapsing and dying. That said another possibility is just years of stagnation and low returns on your investment
I don't invest in stocks so this might be a stupid question: I get that Intel is undervalued now. If things go right it should have a significant premium over its current value. But doesn't that also depend on the rest of the semiconductor sector? Let's say the AI boom bursts. Wouldn't this be a big hit for Intel due to slowdown of semi market, although they are doing good?
It's not a blog post, it's an RFD. We have a strong focus on writing as part of thinking and making decisions, and when we can, we like to publish our decision making documents in the spirit of open source. This is not a defence of our position so much as a record of the process through which we arrived at it. This is true of our other RFDs as well, which you can see on the site there.
> It also should not matter to their customers. They get exposed APIs and don't have to care about the implementation details.
Yes, the whole product is definitely designed that way intentionally. Customers get abstracted control of compute and storage resources through cloud style APIs. From their perspective it's a cloud appliance. It's only from our perspective as the people building it that it's a UNIX system.
So at no point did anyone even suspect that Illumos was under consideration because it's been corporate leadership's pet project for decades? That seems like a wild thing to omit from the "RFD" process. Or were some topics not open to the "RFD" process?
We are trying to build a business here. The goal is to sell racks and racks of computers to people, not build a menagerie of curiosities and fund personal projects. Everything we've written here is real, at least from our point of view. If we didn't think it would work, why would we throw our own business, and equity, and so on, away?
The reason I continue to invest myself, if nothing else, in illumos, is because I genuinely believe it represents a better aggregate trade off for production work than the available alternatives. This document is an attempt to distill why that is, not an attempt to cover up a personal preference. I do have a personal preference, and I'm not shy about it -- but that preference is based on tangible experiences over twenty years!
Furthermore, your team have already demonstrated that you can reevaluate things that you had strong opinions on, and come to a different conclusion. I'm thinking of the decision to exclusively run hardware-virtualized VMs rather than LX-branded zones, as we discussed on Oxide and Friends before.
I don't think working at Oxide would be for me, but I respect the team's values and process.
It's not his code that people felt hurt by, was it?