Depends how good it is now, doesn’t it? If it actually writes code (it does), which is ‘good enough’ compared to a 25-150$/hr human (it does for the lower part of that scale), the $2 is definitely a good business case. For instance, I had to write 1500 lines of js yesterday because one of my colleagues who took the task could not manage it in the previous 2 days and the deadline was today. We would’ve saved about 500$ in total doing it with that expensive API from the start. Now it was free besides my hourly wage (it took me little over an hour) so it made even more sense.
> which is ‘good enough’ compared to a 25-150$/hr human (it does for the lower part of that scale),
I'm not exactly sure you define that scale, perhaps Minecraft bots or the like, where the damage after complete failure is self contained to perhaps a few dollars or a few hours of human annoyance. I'm sure there are many niches where a 50% success rate of mass generated programs can earn you big bucks.
But in my experience Codex does very limited reasoning about code paths. For the current state of the art, you are almost guaranteed to have catastrophic bugs in any non-trivial programs engineered by prompt.
When a bad senior or junior/medior delivers their pr, I (or some other skilled senior) review it and it has issues which I explain and we go in a fix loop. Often I just approve it and fix it myself as it takes too long. That is good enough as that’s going on in all companies. Gpt is the same only many times faster; the loop is instant and sometimes I give up and fix it myself, as it is not going to get it, just like some (depressingly many) humans.
