I think it also really accelerates learning of a new language or framework, when that language or framework is really well documented on the web. For novel programming frameworks, obviously it's a bit more challenging to get help from an LLM.
One of more recent attempts at using LLM code assist was to try to fix a bug in a Swift SSH Agent's connection handling that was causing hangs. I know zero Swift, much less the networking frameworks. So I pumped the output of `tree` on the git repo into the LLM, asked for which file likely handled connections, and it found it right away. That's probably 15 minutes saved. Before putting in the file I asked for likely reasons for deadhangs, got that list, then put in the Swift file that handled connections, and it pointed to what the likely problem was. That's probably 1 hour+ of reading documentation to try to figure out what the code was doing wrong with the networking framework, assuming the LLM was not hallucinating. And that "not hallucinating" probability is high enough in my experience that I spend >50% of my time trying to verify I'm not getting bullshitted.
The LLM proposed a fix (~10-20 minute savings), but even as somebody who doesn't use Swift it seemed like >99% chance that it had just introduced a bunch of race conditions in the data structures it used to track connection status. So I asked about it, and it said "Oh yeah of course how could I forget" and then significantly complicated the solution with something that I thought looked like it probably worked. But was the LLM just being obsequious or was it correct the first time? So hard to tell...
So in about 20 minutes I probably accomplished in a language I didn't know, in a code base I didn't know, about 2 hours+ of learning.
But if I knew the language, it would have saved me very little time, and may have cost me some time.
I think it also really accelerates learning of a new language or framework, when that language or framework is really well documented on the web. For novel programming frameworks, obviously it's a bit more challenging to get help from an LLM.
One of more recent attempts at using LLM code assist was to try to fix a bug in a Swift SSH Agent's connection handling that was causing hangs. I know zero Swift, much less the networking frameworks. So I pumped the output of `tree` on the git repo into the LLM, asked for which file likely handled connections, and it found it right away. That's probably 15 minutes saved. Before putting in the file I asked for likely reasons for deadhangs, got that list, then put in the Swift file that handled connections, and it pointed to what the likely problem was. That's probably 1 hour+ of reading documentation to try to figure out what the code was doing wrong with the networking framework, assuming the LLM was not hallucinating. And that "not hallucinating" probability is high enough in my experience that I spend >50% of my time trying to verify I'm not getting bullshitted.
The LLM proposed a fix (~10-20 minute savings), but even as somebody who doesn't use Swift it seemed like >99% chance that it had just introduced a bunch of race conditions in the data structures it used to track connection status. So I asked about it, and it said "Oh yeah of course how could I forget" and then significantly complicated the solution with something that I thought looked like it probably worked. But was the LLM just being obsequious or was it correct the first time? So hard to tell...
So in about 20 minutes I probably accomplished in a language I didn't know, in a code base I didn't know, about 2 hours+ of learning.
But if I knew the language, it would have saved me very little time, and may have cost me some time.