One of my favorite CS professors back in the 80s used to joke that we had gone from cave paintings to written language, and now we wanted to back with our computers. (my paraphrase of whatever he actually said back then)
Pictographs work for the illiterate, but that doesn't make them the most efficient interchange mechanism.
A professor I knew had a good way to put it: "With the shell, you have a language. With pointy-clicky, you're reduced to pointing at things and grunting."
While I enjoy the shallow dig, I think this is a pretty great metaphor generally, in a deep (and less condescending) way.
If you drop me in France, and I don't speak French, I'll probably do a lot of communication with pointing and grunting, and I'll probably be able to accomplish simple tasks (particularly with an ideographic picture book on hand). Expressing more complicated things that way is pretty intractable, however. Learning to speak French is the solution, but it's a lot of work.
Moreover, there are certainly contexts where pointing makes more sense ("I'll take that one, that one, and that one" versus trying to pick out differentiating features or count).
Pictographs work for the illiterate, but that doesn't make them the most efficient interchange mechanism.