Latency will only be an issue if it's non-constant - most gamers can adjust for lag if it's expected.

I think the underlying idea here is consistency. If I point and click on something that was merely a "mirage" due to network effects, then this is somewhat of a bad experience.