People tell us a lot of things, including a lot of contradictory ones.
If it were simply a matter of being told, I could have just told myself, since I've made that argument myself on HN for years. But the point was to try the idea for a little while and see what happened. Now we have actual experience to point to.
Perhaps we're using the word "experiment" differently. I don't mean a controlled experiment, I mean making a change for a little while to see what happens. One doesn't need a precise definition to do that. In fact, if we had one it would have been useless, because in complex social systems like HN such information doesn't transmit. People would have flagged whatever they thought of as 'politics', no matter what we said. Which is what they did.
I won't say it didn't cross my mind. But if you're arguing that everything should have been obvious, that was far from the case. People have very different, shockingly different, way-more-different-than-you'd-expect assumptions about things that seem obvious to you (i.e. any one of us); since our job is to serve the community as a whole, we can't just go by our own. It's a constant struggle.
HN is large enough that if the outcome had been wildly different, other users would be thinking that had been obvious all along, and telling us so.
Keep in mind that the important question was how the community would react, and complex social systems aren't straightforwardly predictable. I've been working with HN for years now, and a participant for years before that, and much of it remains counterintuitive to me.
It's a good idea to think through ideas before attempting them.
This experiment was like asking people to flag "news", and getting surprised that people were flagging tech news, when they really wanted to avoid crime news.