The problem here is to write a program, not solve the problem that this program is intended to solve. Clearly people can write tests for programs without necessarily being able to write a program to meet the specification, and in some cases without being able to solve the problem that the program is required to solve (e.g. plausibly, a person could write a tests for a program for solving Sudoku puzzles without being able to do that themselves, and it is possible to test programs that will be employed to find currently-unknown prime numbers.)
Having said that, your point is kind-of what I was getting at here, though in a way that was probably way too tongue-in-cheek for its own good: When we consider all the activities that go into writing a program, the parts that AlphaCode does not have to do are not trivial. Being given solved test cases is what allows it to succeed (sometimes) with an approach that involves producing a very large number of mostly-wrong candidates, and searching through them for the few that seem to work.
Having said that, your point is kind-of what I was getting at here, though in a way that was probably way too tongue-in-cheek for its own good: When we consider all the activities that go into writing a program, the parts that AlphaCode does not have to do are not trivial. Being given solved test cases is what allows it to succeed (sometimes) with an approach that involves producing a very large number of mostly-wrong candidates, and searching through them for the few that seem to work.