That occurred on Digg for a while, where a group of 'power users' submitted basically all of the content, which turned into a self-reinforcing cycle.
The problem lies when normal people stop thinking that they can submit and get content to the top, the submissions tend to become rote, and you lose community interest.
(As an aside some of the power users on Digg started taking bribes and were banned en-masse IIRC, but I don't see that happening here.)
I agree: I don't think that the theoretical oligarchy should have complete control. But moderation control is different. We might get a more civilized community if the upvotes and downvotes were controlled somehow.
It won't happen, but I think it would be an interesting experiment.
The problem lies when normal people stop thinking that they can submit and get content to the top, the submissions tend to become rote, and you lose community interest.
(As an aside some of the power users on Digg started taking bribes and were banned en-masse IIRC, but I don't see that happening here.)