Note that Bayesian methods also have analytical solutions in some settings.

There is a reason why conjugate priors were a thing.