Potentially, but that doesn't address the problem of replacing writing code with the (harder) process of reading, verifying and fixing it. However well I engineer my inputs I'll still have to review, verify and fix the outputs.
This problem has been (albeit imperfectly) addressed in speech recognition. When error corrections are made through the the UI the engine can learn from those corrections. Presumably over time the corrections needed in alphacode will become more semantic than syntactical. But you're right, correcting subtly flawed code or text is way harder than writing from scratch.
One of the most difficult problems in software development in general is coming up with the right test cases that cover the real-world domain's needs. If you have an AI like this at your disposal, that you can throw test cases at and it will give you the stupidest piece of code that makes them pass, then at the least you have something that helps you iterate on your test cases a lot more effectively, which would be a great boon.
