Atomic weapons emerged slightly before the first craft to leave the Earth’s atmosphere, but we’re still nowhere near interstellar travel. On the other hand, the difficulty of space travel is largely a function of the escape velocity, which is dependent on the gravitational pull of your planet and solar system (both of which are variable).

One issue with your claim though is that once you have interstellar flight, you certainly have weapons of mass destruction. Even crashing a coke-can sized meteor into the Earth at 1% of the speed of light would cause major destruction.