I am frequently bothered by the misuse of the word "universe". It is defined as the totality of all that exists -- that we can observe. So if there is anything out there, which we didn't think existed before, but we now suspect does exist, well... it's just part of our same old universe.
The definition is changing. There's a lot of people who use it to mean something like "all the stuff that resulted from the big bang". Which may or may not be the entirety of existence. If it's not, then the other stuff will be stuff that exists outside of the universe. Which does contradict the etymology of the word but is consistent with how it's being used today. Much like the word atom I think.
I'm not an astrophysicist, so hopefully one can chime in, but my understanding is that 'universe' == the x,y,z,t continuum. Anything outside of that is a different 'universe'.
If you can imagine a place that requires alternate dimensions in order to access (e.g. klein bottle), you would be traveling through a different 'universe' (though it is possible that a passage through other universes could lead back to our own). If, however, the other place has a different x,y,z,t continuum, even if there is a passage between the two (wormhole), it's still a different universe.
EDIT: (Appending this)
Also, if you think of the word universe, it means one totality (uni=one, verse=totality). So, if you want to refer to all that can exist, you probably want to say 'omniverse'.
I think the concept of outside the universe or inside of it are not developed in a sufficiently rigorous way to really define what a universe is. However in some theories objects appear were it is actually quite natural to speak of an universe and something else. For example there is a unique way to extend the Schwartzschild space time [1] such that all geodesics either terminate in the singularity or go to infinity. But if you do this, then a region of the spacetime appears, that is 'on the other side' of the black hole. And there is no way to communicate with this region of spacetime, since all geodesics leading to it are space like, that is they would require (local) superluminal travel. It is therefore sometimes called a parallel universe.
It is similar in certain string theories, that patches of spacetime with different physical laws appear, this is then called the landscape. Or the multiverse in some cosmological models, where a far away region may appear, which is vastly different from or 'local' universe, that is in this case everything we can possibly observe.
So I think that it is quite possible that a theory of quantum gravity will have some object which we will call a universe, but until we have such a theory the appearance of anything outside of the universe marks highly speculative physics.
[1] That is the spacetime describing a black hole.
Are you a fan of Ayn Rand? (I am.) Just curious, as she insists on not calling a concept by a name (word) already taken by something else, which is what you're alleging in this case.
Tangentially, I think it may actually be appropriate to use "universe" to refer to a cosmological phenomenon (as modern physicists do), and "existence" to refer to the metaphysical phenomenon you are describing. I guess I'd have to research the etymology of "universe" before I could take a more definitive stand.
