I've been getting to grips with some Bayesian modelling, but one thing is confusing the heck out of me when I look at tutorials and worked-through problems online. I'm looking at a problem with a Dirichlet prior with alpha length 3, and observed data with a multinomial distribution. So we end up with a Dirichlet posterior. All the examples I look at online use MCMC sampling methods to form the posterior, but it is my understanding that you don't need to sample with conjugate distributions, since the posterior can be solved analytically.
If I'm incorrect, and you do need to sample using conjugate priors, what is happening during each sampling step? Is it sampling one value from the Dirichlet distribution or three? Is it updating the Dirichlet prior with each step?
Thanks for any help.