> remains the same whether they pull source or binaries.
I don't think that's exactly true, it's definitely _easier_ to sneak something into a binary without people noticing than it is to sneak it into rust source, but there hasn't been an underhanded rust competition for a while so I guess it's hard to be objective about that.
Pretty much nobody does those two things at the same time:
- pulling dependencies with cargo
- auditing the source code of the dependencies they're building
You are either censoring and vetting everything or you're using dependencies from crates.io (ideally after you've done your due diligence on the crate), but should crates.io be compromised and inject malware in the crates' payload, I'm ready to bet nobody would notice for a long time.
I fully agree with GP that binary vs source code wouldn't change anything in practice.
> Pretty much nobody does those two things at the same time:
- pulling dependencies with cargo - auditing the source code of the dependencies they're building
Your “pretty much” is probably weaseling you out of any criticism here, but I fully disagree:
My IDE (rustrover) has “follow symbol” support, like every other IDE out there, and I regularly drill into code I’m calling in external crates. Like, just as often as my own code. I can’t imagine any other way of working: it’s important to read code you’re calling to understand it, regardless of whether it’s code made by someone else in the company, or someone else in the world.
My IDE’s search function shows all code from all crates in my dependencies. With everything equal regardless of whether it’s in my repo or not. It just subtly shades the external dependencies a slightly different color. I regularly look at a trait I need from another crate, and find implementations across my workspace and dependencies, including other crates and impls within the defining crate. Yes, this info is available on docs.rs but it’s 1000x easier to stay within my IDE, and the code itself is available right there inline, which is way more valuable than docs alone.
I think it’s insane to not read code you depend on.
Does this mean I’m “vetting” all the code I depend on? Of course not. But I’m regularly reading large chunks of it. And I suspect a large chunk of people work the way I do; There are a lot of eyeballs on public crates due to them being distributed as source, and this absolutely has a tangible impact on supply chain attacks.
> Does this mean I’m “vetting” all the code I depend on? Of course not.
Inspecting public facing parts of the code is one thing, finding nasty stuff obfuscated in a macro definition or in a Default or Debug implementation of a private type that nobody is ever going to check outside of auditors is a totally different thing.
> My IDE (rustrover) has “follow symbol” support
I don't know exactly how it works for RustRover, since I know Jetbrain has reimplemented some stuff on their own, but if it evaluates proc macros (like rust-analyzer) does, then by the time you step into the code it's too late, proc macros aren't sandboxed in any ways and your computer could be compromised already.
The point of my argument is not to say I’m vetting anything, but to say that there are tons of eyeballs on crates today, because of the fact that they are distributed as source and not a binary. It’s not a silver bullet but every little bit helps, every additional eyeball makes hiding things harder.
The original claim is that “pretty much no one” reads any of their dependencies, in order to support a claim that they should be distributed as binaries, meaning “if there was no source available at all in your IDE, it wouldn’t make a difference”, which is just a flatly wrong claim IMO.
A disagreement may be arising here about the definition of “audit” vs “reading” source code, but I’d argue it doesn’t matter for my point, which is that additional eyeballs matter for finding issues in dependencies, and seeing the source of your crates instead of a binary blob is essential for this.
> The original claim is that “pretty much no one” reads any of their dependencies,
No the claim is that very few people read the dependencies[1] enough to catch a malicious piece of code. And I stand by it. “Many eyeballs” is a much weaker guarantee when people are just doing “go to definition” from their code (for instance you're never gonna land on a build.rs file this way, yet they are likely the most critical piece of code when it comes to supply chain security).
[1] (on their machines, that is if you do that on github it doesn't count since you have no way to tell it's the same code)
> No the claim is that very few people read the dependencies[1] enough to catch a malicious piece of code.
You’re shifting around between reading enough to catch any issue (which I could easily do if a vulnerability was right there staring at me when I follow symbol) to catching all issues (like your comment about build.rs.) Please stick with one and avoid moving goal posts around.
There exists a category of dependency issues that I could easily spot in my everyday reading of my dependencies’ source code. It’s not all of them. Your claim is that I would spot zero of them, which is overly broad.
You’re also trying to turn this into a black-or-white issue, as if to say that if it isn’t perfect (ie. I don’t regularly look at build.rs), it isn’t worth anything, which is antithetical to good security. The more eyeballs the better, and the more opportunities to spot something awry, the better.
I'm not moving the goal post, a supply chain attack is an adversarial situation it is not about spotting an issue occurring at random, it is about spotting an issue specially crafted to avoid detection. So in practice you are either able to spot every kind of issues, or none of the relevant ones because if there's one kind that reliably slips through, then you can be certain that the attacker will focus on this kind and ignore the trivial to spot ones.
If anything, having access to the source code gives you an illusion of security, which is probably the worse place to be in.
The worse ecosystem when it comes to supply chain attacks is arguably the npm one, yet there anyone can see the source and there are almost two orders of magnitude more eyeballs.
In such an environment I’m doomed anyway, even if I’m vetting code. I don’t understand why the goal has to be “the ability to spot attacks specifically designed to prevent you from detecting.” For what you’re describing, there seems to be no hope at all.
It’s like if someone says “don’t pipe curl into bash to install software”, ok that may or may not be good advice. But then someone else says “yeah, I download the script first and give it a cursory glance to see what it’s doing”, wouldn’t you agree they’re marginally better off than the people who just do it blindly?
If not, maybe we just aren’t coming from any mutual shared experience. It seems flatly obvious to me that being able to read the code I’m running puts me in a better spot. Maybe we just fundamentally disagree.
> It’s like if someone says “don’t pipe curl into bash to install software”, ok that may or may not be good advice. But then someone else says “yeah, I download the script first and give it a cursory glance to see what it’s doing”, wouldn’t you agree they’re marginally better off than the people who just do it blindly?
I don't agree with your comparison, in this case it's more like downloading, then running it without having read it and then every once in a while look at a snippet containing a feature that interest you.
The comparison to “download the script and read it before you run it” would be to download the crate's repo, read it and then vendor the code you've read to use as a dependency, which is what I'd consider proper vetting (in this case the attacker would need to be much more sophisticated to avoid detection, it's still possible but in this case at least you've actually gained something), but it's a lot more work.
> Domain discovery is so underrated. LLMs remove so much friction that makes everything so incredibly accessible.
And, unfortunately, a lot of friction from not having access to information in the first place. I've read a bunch of docs from people talking to Glean in order to explore a new topic; if it's a topic I'm actually very familiar with then four out of five times it is somewhere from misleading to catastrophically wrong. Any internal terminology that doesn't match the common usage outside of our organization poisons the whole session and it will make up things to join the meanings together, and the prompter is none the wiser.
I trust AI only as a gap filler in domains that I'm already an expert in or where there's little internal context, anything else is intellectual suicide.
I feel like if you're using it in the right way, asking the AI to write code, or give domain context in a specific way then the answers it gives are easily verifiable enough- and it's domain knowledge you wouldn't have gotten very easily through a series of google searches- LLMs as a kind of search can work great.
> Now reuse the same connection to request the nested data, which can all have more nested links in them, and so on.
This still involves multiple round-trips though. The approach laid out in the article lets you request exactly the data you need up-front and the server streams it in as it becomes available, e.g. cached data first, then data from the DB, then data from other services, etc.
When you have an HTTP/2 connection already open a 'round-trip' is not really a gigantic concern performance-wise. And it gives the client application complete control and ver what nested parts it wants to get and in what order. Remember that the article said it's up to the server what order to stream the parts? That might not necessarily be a good idea on the client side though. It would probably be better for the client to decide what it wants and when. Eg, it can request the header and footer, then swap in a skeleton facade in the main content area, then load the body and swap it in when loaded.
Round trips for parallel requests work fine over HTTP/2. (As long as there aren't vast numbers of tiny requests, for example every cell in a spreadsheet).
However, sequentially-dependent requests are about as slow with HTTP/2 as HTTP/1.1. For example, if your client side, after loading the page, requests data to fill a form component, and then that data indicates a map location, so your client side requests a map image with pins, and then the pin data has a link to site-of-interest bubble content, and you will be automatically expanding the nearest one, so your client side requests requests the bubble content, and the bubble data has a link to an image, so the client requests the image...
Then over HTTP/2 you can either have 1 x round trip time (server knows the request hierarchy all the way up to the page it sends with SSR) or 5 x round trip time (client side only).
When round trip times are on the order of 1 second or more (as they often are for me on mobile), >1s versus >5s is a very noticable difference in user experience.
With lower latency links of 100ms per RTT, the UX difference between 100ms and 500ms is not a problem but it does feel different. If you're on <10ms RTT, then 5 sequential round trips are hardly noticable, thought it depends more on client-side processing time affecting back-to-back delays.
Assuming a stable connection, there is no meaningful performance difference between a request/response round-trip from the client to the server, and a response streamed from the server to the client, amortized over time.
The article doesn't advocate sending it progressively to make it smaller on the wire.
The motivating example is one where some of the data (e.g. posts) is available before the rest of the data in the response (e.g. comments). Rather than:
- Sending a request for posts, then a request for comments, resulting in multiple round trips (a.k.a. a "waterfall"), or,
- Sending a request for posts and comments, but having to wait until the commends have loaded to get the posts,
...you can instead get posts and comments available as soon as they're ready, by progressively loading information.
The message, though, is that this is something a full-stack web framework should handle for you, hence the revelation at the end of the article about it being a lesson in the motivation behind React's Server Components.
> Why make source available at all if you are just going to release proprietary software?
Because there's years between one decision and the other, and in that time it turned out that because the software was open source they were structurally disadvantaged versus large cloud providers in being able to make money from it.
It seems disingenuous to suggest that the open sourcing was in bad faith given it happened years and paradigms apart from the closed sourcing, even though I don't agree with the decision either.
> It seems disingenuous to suggest that the open sourcing was in bad faith given it happened years and paradigms apart from the closed sourcing, even though I don't agree with the decision either.
The problem here is that their actions are indistinguishable from an entity that is acting in bad faith. "Get loads of free labour, then claim ownership of it and try to monetize" is a business strategy.
Safari is too valuable of a platform for web developers to ignore, but otherwise yes it's the only real exception to the Chrome monopoly and still certainly much smaller in terms of absolute users.
It's barely one step removed from just being burned in an incinerator, it's maybe recycling in a very strict sense but it's definitely misleading to say that.
Easy access to the videos without having to download them from Google (and without Google trying to stop you from scraping them, which they will) is an enormous advantage. There's way, way too much on Youtube for to index and use over the internet, and especially not at full resolution.
I'm on a four year old mid-tier laptop and opening VS Code takes maybe five seconds. Opening IDEA takes five seconds. Opening twitter on an empty cache takes perhaps four seconds and I believe I am a long way from their servers.
On my work machine slack takes five seconds, IDEA is pretty close to instant, the corporate VPN starts nearly instantly (although the Okta process seems unnecessarily slow I'll admit), and most of the sites I use day-to-day (after Okta) are essentially instant to load.
I would say that your experiences are not universal, although snappiness was the reason I moved to apple silicon macs in the first place. Perhaps Intel is to blame.
VS Code defers a lot of tasks to the background at least. This is a bit more visible in intellij; you seem to measure how long it takes to show its window, but how long does it take for it to warm up and finish indexing / loading everything, or before it actually becomes responsive?
Anyway, five seconds is long for a text editor; 10, 15 years ago, sublime text loaded and opened up a file in <1 second, and it still does today. Vim and co are instant.
Also keep in mind that desktop computers haven't gotten significantly faster for tasks like opening applications in the past years; they're more efficient (especially the M line CPUs) and have more hardware for specialist workloads like what they call AI nowadays, but not much innovation in application loading.
You use a lot of words like "pretty close to", "nearly", "essentially", but 10, 20 years ago they WERE instant; applications from 10, 20 years ago should be so much faster today than they were on hardware from back then.
I wish the big desktop app builders would invest in native applications. I understand why they go for web technology (it's the crossplatform GUI technology that Java and co promised and offers the most advanced styling of anything anywhere ever), but I wish they invested in it to bring it up to date.
I disagree. Vs code uses plugins for all its heavy lifting. Even a minimal plugin setup is substantially slower to load than sublime is, which can also have an LSP plugin.
>Anyway, five seconds is long for a text editor; 10, 15 years ago, sublime text loaded and opened up a file in <1 second, and it still does today. Vim and co are instant.
Do any of those do the indexing that cause the slowness? If not it's comparing apples to oranges.
> You use a lot of words like "pretty close to", "nearly", "essentially", but 10, 20 years ago they WERE instant; applications from 10, 20 years ago should be so much faster today than they were on hardware from back then.
11 years ago I put in a ticket to slack asking them about their resource usage. Their desktop app was using more memory than my IDE and compilers and causing heap space issues with visual studio. 10 years ago things were exactly the same. 15 years ago, my coworkers were complaining that VS2010 was a resource hog compared to 10 years ago. My memory of loading photoshop in the early 2000’s was that it took absolutely forever and was slow as molasses on my home PC.
I don’t think it’s necessarily gotten worse, I think it’s always been pathetically bad.
"early 2000s" was at least 22 years ago, as well. Sorry if this ruins your night. 100mhz 1994 vs 1000mhz in 2000, that's the only parallel i was drawing. 10x faster yet somehow adobe...
Ah sorry - I’m in my mid 30s so my early pc experiences as a “power user” were win XP, by which point photoshop had already bolted on the kitchen sink and autodesk required a blood sacrifice to start up.
5 seconds is a lot for a machine with an M4 Pro, and tons of RAM and a very fast SSD.
There's native apps just as, if not more, complicated than VSCode that open faster.
The real problem is electron. There's still good, performant native software out there. We've just settled on shipping a web browser with every app instead.
There is snappy electron software out there too, to be fair. If you create a skeleton electron app it loads just fine. A perceptible delay but still quick.
The problem is when you load it and then react and all its friends, and design your software for everything to be asynchronous and develop it on a 0 latency connection over localhost with a team of 70 people where nobody is holistically considering “how long does it take from clicking the button to doing the thing I want it to do”
It's probably more so that any corporate Windows box has dozens of extra security and metrics agents interrupting and blocking every network request and file open and OS syscall installed by IT teams while the Macs have some very basic MDM profile applied.
This is exactly it. My Debian Install on older hardware than my work machine is relatively snappy. The real killer is the Windows Defender Scans once a week. 20-30% CPU usage for the entire morning because it is trying to scan some CDK.OUT directory (if I delete the directory, the scan doesn't take nearly as long).
This is my third high end workstation computer in the last 5 years and my experience has been roughly consistent with.
My corporate vpn app is a disaster on so many levels, it’s an internally developed app as opposed to Okta or anything like that.
I would likewise say that your experience is not universal, and that in many circumstances the situation is much worse. My wife is running an i5 laptop from 2020 and her work intranet is a 60 second load time. Outlook startup and sync are measured in minutes including mailbox fetching. You can say this is all not the app developers fault, but the crunch that’s installed on her machine is slowing things down by 5 or 10x and that slowdown wouldn’t be a big deal if the apps had reasonable load times in the first place.
I don't think that's exactly true, it's definitely _easier_ to sneak something into a binary without people noticing than it is to sneak it into rust source, but there hasn't been an underhanded rust competition for a while so I guess it's hard to be objective about that.
reply