Sounds like they had some fun, but what is missing from the analysis is the effort spent, or even a rough indication of said effort. The epilog lists about sixteen non-security contributors that were needed to achieve the feat, and probably many more security personnel were needed from GPZ. Yet the outcome is one bug that could have been found with manual code review or a custom fuzzer. I am not sold on the premise that LLMs can increase code security or lower costs. At the moment, LLMs are producing and introducing more insecure code than they are finding.
This seems both pessimistic and conflating two issues. It's entirely possible that generating code is a bad idea but LLMs can be effective at finding vulnerabilities in human written code. Also, this is an applied research project, you have start small to prove the idea and iron out the wrinkles then you can find efficiencies to scale up later.
I agree it is a pessimistic view. I did not mean to make a disparaging comment, though. I agree that all research starts out small. And, naturally, for any influential idea, you can point back to when it was small.
I did not intend to conflate separate issues. I agree assisted coding and assisted bug hunting can coexist. I was merely trying to weigh the net effect LLMs have on security.
