Yes, it's definitely vulnerable to situations where a large number of people have been "seeded" with the specific belief that they are part of a minority that holds the correct answer.
Still, the article says this:
Because of the relatively high margin of 10%, there can
be high confidence that the [surprisingly popular answer
is correct]
I would maintain this holds true, even though we can name some easy counterexamples like flat-earthism.
One, I'm sure that nowhere near 10% of an educated population believes in flat-earthism.
Two, I think this technique is certain to yield the correct answer quite a bit more often than it will yield an incorrect one. Since it claims to achieve "high confidence" and not "absolute confidence" I think it would still be a pretty valuable metric in many instances.
Still, the article says this:
I would maintain this holds true, even though we can name some easy counterexamples like flat-earthism.One, I'm sure that nowhere near 10% of an educated population believes in flat-earthism.
Two, I think this technique is certain to yield the correct answer quite a bit more often than it will yield an incorrect one. Since it claims to achieve "high confidence" and not "absolute confidence" I think it would still be a pretty valuable metric in many instances.