Having worked in this area, I can say you are on the right track. Not only the US, there are many countries/powers that will get very upset if certain people see certain areas.
As for who gets to see what, there are extremely stringent rules set by many of these governments; even within government organizations themselves, there is huge variation in access constraints.
This seems strange to me, considering there is high resolution "birds eye"(taken from a plane) imagery of a variety of sensitive US installations available on Bing, not to mention what's on Google Maps/Earth. There are other sources as well.
As for who gets to see what, there are extremely stringent rules set by many of these governments; even within government organizations themselves, there is huge variation in access constraints.