This is an oversimplification as well. You can use the telephoto lens if the subject has no reasonable expectation of privacy. For example, imagine a privately-owned soccer field that is visible/accessible to everyone. You could stand on a neighboring (public) hill and use a telephoto lens to photograph the players. You could not stand on that same hill and use the telephoto to peer into someone's 5th floor window -- different expectation of privacy.
This is a tremendous oversimplification. For just one example, you can't use a telephoto lens to survey private property.