From a Bayesian practitioner's standpoint, what are the specific conditions to decide on whether we have conjugacy or not?

Question

I've been learning Bayesian statistical analysis on my spare time using textbooks, videos on YT, etc. I'm slowly going up that mountain. Please correct me if my wording below is poor or ask for clarification if it doesn't make sense.

I understand that conceptually you can use an analytically closed form if you "know" (somehow) that you have conjugacy - that is, when the posterior distribution is in the same family as the prior distribution. If we do not know, or if we know that the prior and posterior have different families, we would need to rely on MCMC techniques.

I'm a practitioner though, so this conceptual explanation is unhelpful to me. Concretely, how does one evaluate whether we have conjugacy or not? Or does a practitioner just avoid making that decision nowadays, and ends up routinely using MCMC techniques regardless?

Answer 1

Conjugate prior is an algebraic convenience and nothing more.

It is immediate to calculate the the posterior distribution of a Dirrechlet distribution, when the likelihood function is categorical.

Where as, if you assume the prior distribution is, for example, also categorical, you're either in for a world of pain and complexity calculating all those sums/integrals, or you're gonna need simulation techniques (MCMC).

Think of the a Conjugate prior as an algorithmic optimization and not as a the true descriptive distribution.

And like any other mathematical concept, you first make an assumption, build a model, get the result, and if the result defies your assumption, then it doesn't hold.

Same goes for statistics, in practice, you start by assuming the Conjugate prior, and if that does not describe your data well, you throw it away and go for the big guns (MCMC)