Japan actually has about 3.25× California’s population. OTOH, California has a lot more land area that is, say, desert than Japan does, so there’s that (it also has a lot more that is controlled by a separate sovereign; 47.7% of California is federal land.)
The federally-owned land in California is mostly deserts and mountains.
Besides, even if we're generous and say that half of California is too mountainous to live in and another quarter is too desert-y to live in, leaving only a quarter of the state's area suitable for habitation, that leaves roughly 40 million people living in roughly 40,000 square miles, or about a thousand heads per square mile. That's comparable to Israel or Belgium in terms of population density, and certainly not as though people are packing into Kowloon.
