I think the more important question is should you? If 10 other people before have expended the mental energy to figure out that this comment isn't worth reading, why should I expend mine to determine the same thing? I like where you're going with this, but it uses our group mental capacity much less efficiently. There is probably a better solution... looking at a number is a pretty efficient filter and I don't think we should lose it.
In most cases you’re probably right. But not always.
The problem is you don’t know the moral character or intelligence level of the people who first stumbled on to a comment. If two ignorant people happen to see a quality comment first and both vote it down that comment is essentially lost to the group. I think we can all agree there are at least a few fools running around here so the idea isn’t farfetched.
So the question is do you want to trust what might be insightful comments to the randomness of timing?
If the comment was really sparkling, it seems unlikely that the first two ignoramuses would drown out the larger cast of HN. It's not all about timing.
What if we don't assume every vote is equal. What about weighting a user's vote by some factor of their karma? And/or capping the threshold that a user can vote. i.e A user with 0 karma can vote on comments with 0-3 points. A person with 100 karma can vote on comments -3-6 points. A person with 500 karma can vote on comments -9-9 points, etc... This implies that a comment with ridiculous votes (either positive or negative) has been decided by members of the community that the community has already shown approval for.