I took a class in college on voice interface design, and the professor named this as the biggest hurdle that's yet to be overcome. Screens and keyboards are the computer's domain, and we're comfortable adapting to them. But speech is our domain, and we expect the computer to adapt to us. The problem is that the tech isn't quite there for real understanding of language.