I'm surprised to see a huge disconnect between how I perceive things and the vast majority of comments here.
AI is obviously not good enough to replace programmers today. But I'm worried that it will get much better at real-world programming tasks within years or months. If you follow AI closely, how can you be dismissive of this threat? OpenAI will probably release a reasoning-based software engineering agent this year.
We have a system that is similar to top humans at competitive programming. This wasn't true 1 year ago. Who knows what will happen in 1 year.
Nobody can tell you whether progress will continue at current, faster or slower rates - humans have a pretty terrible track record at extrapolating current events into the future. It's like how movies in the 80's made predictions about where we'll be in 30 years time. Back to the Future promised me hoverboards in 2015 - I'm still waiting!
Compute power increases and algorithmic efficiency improvements have been rapid and regular. I'm not sure why you thought that Back to the Future was a documentary film.
Unless you have a crystal ball there is nothing that can give you certainty that will continue at the same or better rate. I’m not sure why you took the second half of the comment more seriously than the first.
Why do you think this is a fundamental hurdle, rather than just one more problem that can be solved? I dont have strong evidence either way, but I've seen a lot of 'fundamental unsurmountable problems' fall by the wayside over the past few years. So I'm not sure we can be that confident that a problem like this, for which we have very good classic algorithms, is a fundamental issue.
This is not about the numerical result, but about the way it reasons. Testing is a sanity check, not a substitute for reasoning about program correctness.
It's the opposite. I don't think it'll replace programmers legitimately within a decade. I DO think that companies will try a lot in the months and years anyway and that programmers will be the only ones suffering the consequences of such actions.
People somehow have expectations that are both too high and too low at the same time. They expect (demand) current language models completely replace a human engineer in any field without making mistakes (this is obviously way too optimistic) while at the same time they are ignoring how rapid the progress has been and how much these models can now do that seemed impossible just 2 years ago, delivering huge value when used well, and they assume no further progress (this seems too pessimistic, even if progres is not guaranteed to continue at the same rate).
Really now. I think that deserves a bit more explaination, given the cost per token has dropped by several orders of magnitude, we have seen large changes on all benchmarks (including entirely new capabilities), multimodality is now a fact since 4o, test time compute with reasoning models is making big strides since o1.... It seems on the surface a lot is happening. In fact, I wanted to share one of the benchmark overviews, but none include ChatGPT 4 anymore since it is totally not competitive anymore..
Benchmarks are meaningless in and of themselves, they are supposed to be a proxy for usefulness. I have used Sonnet 3.5, ChatGPT-3, ChatGPT-3.5, ChatGPT-4, ChatGPT-4o, o1, o3-mini, o3-mini-high nearly daily for software development. I am not saying AI isn't cool or useful but I am experiencing diminishing returns in model quality (I do appreciate the cost reductions). The sorts of things I can have AI do really haven't changed that much since I got access to my first model. The delta between having no LLM to an LLM feels an order of magnitude bigger at least than the delta between the first LLM and now.
depends on what you work on in the software field. Many of these LLM’s have pretty small context windows. In the real world when my company wants to develop a new feature, or change the business logic, that is a cross-cutting change (many repos/services). I work at a large org for background. No LLM will be automating this for a long time to come. Especially if you’re in a specific domain that is niche.
If your project is very small, and it’s possible to feed your entire code base into an LLM in the near future, then you’re in trouble.
Also the problem is the LLM output is only as good as the prompt. 99% of the time the LLM won’t be thinking of how to make your API change backwards compatible for existing clients, how to help you do a zero-downtime migration, following security best practices, or handling a high volume of API traffic. Etc.
Not to mention, what the product team _thinks_ they want (business logic) is usually not what they really want. Happens ALL THE TIME friend. :) It’s like the offshoring challenge all over again. Communication with humans is hard. Communication with an LLM is even harder. Writing the code is the easiest part of my job!
I think some software development jobs will definitely be at risk in the next 10-15 years. Thinking this will happen in 1 years time is myopic in my opinion.
> If you follow AI closely, how can you be dismissive of this threat?
Just use a state of the art LLM to write actual code. Not just a PoC or an MVP, actual production ready code on an actual code base.
It’s nowhere close to being useful, let alone replacing developers. I agree with another comment that LLMs don’t cut it, another breakthrough is necessary.
It's a bit paradoxical. A smart enough AI, and there is no point in worrying, because almost everyone will be out of a job.
The problem case is the somewhat odd scenario where there is an AI that's excellent at software dev, but not most other work, and we all have to go off and learn some other trade.
AI is obviously not good enough to replace programmers today. But I'm worried that it will get much better at real-world programming tasks within years or months. If you follow AI closely, how can you be dismissive of this threat? OpenAI will probably release a reasoning-based software engineering agent this year.
We have a system that is similar to top humans at competitive programming. This wasn't true 1 year ago. Who knows what will happen in 1 year.