Pretty much. There's actually quite a bit of overlap between the two. If you're doing statistical work then R is a natural. For engineering tasks Octave/Matlab is more convenient, particularly if you're doing a lot of linear algebra, where Octave's syntax is ideal. You'll also find more third-party code available if you stick to the language closest to your problem domain.

R's graphing facilities are superb though, and easily outdistance Octave's. So if you want to build complex visualizations that might trump other concerns.

Numpy/Scipy can do most of what R and Octave can do but their syntax isn't optimal, since they're built on a general-purpose language.