Indeed this point is often overlooked.

I know the soviets dug up half the Czech Republic for uranium deposits though. There's still some left there, not sure how much though. I have a feeling that the reliance on Africa and Russia is more price and environmental regulation driven.