The tl;dr is it sort of lets you write a program that describes sampling from your data-generating model, perhaps setting priors over certain unknown parameters.
by allowing for programming language constructs, you can define very rich and interesting data-generating distributions.
one can then execute the program to generate samples. more importantly, given observed data, one can efficiently "use Bayes" to update the priors.
this is usually computationally nontrivial due to the Bayes rule denominator aka "partition function", but there are various good algorithms to do it approximately. One of the most popular (used in Stan) is known as Hamiltonian Monte Carlo.
I'd add that sampling is still very useful in contexts when the partition function is known or efficient to calculate.
What's generally interesting is the general character of the posterior -- its modes, the shape of variance around these modes etc. High-quality sampling helps with this, whereas being about to calculate the PDF exactly at particular points sometimes does not.
by allowing for programming language constructs, you can define very rich and interesting data-generating distributions.
one can then execute the program to generate samples. more importantly, given observed data, one can efficiently "use Bayes" to update the priors.
this is usually computationally nontrivial due to the Bayes rule denominator aka "partition function", but there are various good algorithms to do it approximately. One of the most popular (used in Stan) is known as Hamiltonian Monte Carlo.