Of more than a dozen friends who moved to Silicon Valley for work/education, all but one left California around the time they started their families. The impression I get is that they're happy they had a chance to work there for a decade or so and now they're happy to live somewhere else. Silicon Valley sounds like a great place to do many things and it's OK if having a family is not one of those things. I'm not saying it's not possible -- one of my friends is making a go of it. But it does sound like having a family there is playing on Hard Mode.
Contrast that to NYC where most of the folks I know who started families there still work in the city, though some did move a little further out into the suburbs.
When I ran the numbers on some offers years ago my estimate was that moving to Silicon Valley would be a financially sound decision if I lived frugally because I'd be less exposed to the high cost of living. But once you start factoring in things like owning a home, childcare, etc the salary premium compared to cities like NYC, Boston, Seattle, Portland, etc just isn't sufficient.
So it makes some sense that the area attracts young people looking to strike it rich but pressures them to leave if they haven't won the startup lottery and still want to have a family.
Contrast that to NYC where most of the folks I know who started families there still work in the city, though some did move a little further out into the suburbs.