I get the gist of sampling methods in probability. These algorithms were developed while building the Atom Bomb to estimate some distribution. The idea was just to try a simulation and note the results. And keep doing it for long enough, you will get a distribution over the outcomes, that is your probability.
Now, I am really having trouble understanding how do these methods work in machine learning.
Consider a topic model.
This is a probabilistic model like $P(x, z, y)$, where z is the hidden variable and there are distributions linking $x$ y and $z$.
The model is generative so the output is only on $x$ and $y$.
How does sampling method help here? How are the simulations run?
Could you explain how MCMC and Metropolis-Hastings are related to these? How do they actually work, what do they compute? How can you compute them without any optimization?
I really want to understand these methods and try some probabilistic programming but I am not getting the hang of it.