If an AI can do your test better than a human in 2025 it reflects not much better on your test than if a pocket calculator could do your test better than a human in 1970.
That did happen and the result from the test creators was the same back then: "we're not the problem, the machines are the problem. ban them!"
In the long run it turned out that if you could cheat with a calculator though, it was just a bad test....
I think there is an unwillingness to admit that there is a skill issue here with the test creators and that if they got better at their job they wouldnt need to ban candidates from using AI.
It's surprising to hear this from anthropic though.
That did happen and the result from the test creators was the same back then: "we're not the problem, the machines are the problem. ban them!"
In the long run it turned out that if you could cheat with a calculator though, it was just a bad test....
I think there is an unwillingness to admit that there is a skill issue here with the test creators and that if they got better at their job they wouldnt need to ban candidates from using AI.
It's surprising to hear this from anthropic though.