I would be shocked if hormones didn't have some effect on the brain considering how much of an effect they have everywhere else. Hell, I know some people who take the "brain sex" angle re. transness speculate that varying levels of hormonal exposure in utero could be what causes gender dysphoria post-utero... in which case, would adult endogenous hormones be responsible for brain structures, or would brain structure dictate which kind of hormones your body expects?
I think, in utero, the effects would be strong and likely visible, but through childhood, puberty, and then young adulthood, I would expect diminishing degrees of impact. Certainly, I've read case studies of various pre-natal exposures to particular chemicals having large and unmistakable impacts on sexual orientation. It's of especial interest to me as one of the very last DES babies.
Brain structure, as in the gross features visible on imaging, would probably not dictate what kinds of hormones you body might expect. I think that sort of thing would occur at a much lower level, right on the receptors of the tissues, and would not be visible.
