> ... and you trust the dependency author ... it's only a small step to auto-merge
I disagree for the reasons listed above, but let's focus for a moment on 3rd-party dependencies here, versus trusted ones. Given the numerous scenarios I listed above, it's a huge step from "using a 3rd party library" to "trusting the author of it".
We'd have to start with "...and you don't trust the author", because for most 3rd-party dependencies, the author has given us neither sufficient evidence to do so, nor sufficient recourse if that trust is violated.
> Relying on a human reviewer, regardless, is a weak guarantee.
Relying on countless random 3rd party to not own you when they know people are pulling in their code as a dependency is a far, far worse guarantee. How would that strategy have protected someone against this supply-chain attack?:
For most projects (that don't comprehensively review their entire dependency graph's codebases), whether you started with foobar-1.0.1 or foobar-1.0.2 is just a coincidence of timing. They downloaded whatever the current version was when first needing foobar. Selecting 1.0.1 with little review but then declaring that all upgrades deserve deep review is a contradiction. I think it is this contradiction that leads people to auto-update dependencies.
The problem isn't the auto-merging 1.0.2 - it's the lack of attestation in the PR that appears to be code from the foobar authors who were trusted in version 1.0.1.
Why would you choose to give little review to a dependency?
> The problem isn't the auto-merging 1.0.2 - it's the lack of attestation in the PR that appears to be code from the foobar authors who were trusted in version 1.0.1.
The authors were never trusted and never will be. What was trusted was the code at the commit hash for the first time 1.0.1 got tagged, and now a bot is saying you should move away from that trusted code.
Is that a good idea? That depends on, among other things: if you need to; and if you trust the new code.
The foobar author could have intentionally or unintentionally included an exploit, or they could have had their account hacked and someone else included one on their behalf, or just changed the code behind an existing tag (see my previous comments for a recent example).
I get what you're saying about this being overwhelming. Without this, though, we've seen it's just security theater, because your code is as strong as it's weakest link. More eyeballs on a given release/commit also means more people looking out for something nefarious, which is counteracted by a shorter time-since-release. Maybe multiple AI agents will make it easier
> Why would you choose to give little review to a dependency?
Given finite developer hours, what activity has the highest security impact per hour - is it reviewing the dependency graph? That's not the choice most projects are making. Maybe they are wrong. Or maybe they know where to spend their time for better impact. I dunno...
I have worked in commercial codebases that vendored 100% of their dependencies (including kernel and driver source) and reviewed large swathes of those dependencies carefully. I'm absolutely not dismissive of this. I think we agree more people should be doing it.
However, over the decases, I've seen very few projects take this approach. Many choose to trust third party code (naively, as you point out!). If that's the reality, I think we should continue to work on improving provenance, automated signature verification, and other tooling so we can at least better know that if we choose to trust foobar, it's actually foobar who is distributing foobar 1.0.2.
The AI comment is provocative - can future-AI find vulnerabilities better than future-AI can inject hard-to-find vulnerabilities? And how do we know our AI reviewers themselves aren't hacked... a horrible twist on https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref....
I disagree for the reasons listed above, but let's focus for a moment on 3rd-party dependencies here, versus trusted ones. Given the numerous scenarios I listed above, it's a huge step from "using a 3rd party library" to "trusting the author of it".
We'd have to start with "...and you don't trust the author", because for most 3rd-party dependencies, the author has given us neither sufficient evidence to do so, nor sufficient recourse if that trust is violated.
> Relying on a human reviewer, regardless, is a weak guarantee.
Relying on countless random 3rd party to not own you when they know people are pulling in their code as a dependency is a far, far worse guarantee. How would that strategy have protected someone against this supply-chain attack?:
https://www.wiz.io/blog/github-action-tj-actions-changed-fil...