Assigning a scale mixture of Normal distributions for the data

Question

it's well known that the scale mixture of normal distributions is equivalent to a Student t model, that is $$ t_{(v)}(x|\mu,\sigma^2)=\int_0^\infty N(x|\mu,\sigma^2/\lambda)\times G(\lambda|v/2,v/2)d\lambda. $$

a full Bayesian analysis is completed by assigning prior distributions for $\mu$ and $\sigma^2$, say $$ \mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta), $$ where $\mu_0,\sigma_0^2,\alpha,\beta, v$ known.

Considering a random sample $(x_1,...x_n)$, modeled according to the following models:

Model I: $(x_1,...x_n)|\mu,\sigma^2\sim t_{(v)}(\mu,\sigma^2)$

$\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$

Model II: $(x_1,...x_n)|\mu,\sigma^2,\lambda\sim N(\mu,\sigma^2/\lambda)$

$\lambda\sim G(\lambda|v/2,v/2)$

$\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$

When I simulate from the posterior distribution from each model, the posterior estimates for $\mu$ and $\sigma^2$ are not the same in the two models. I was expecting to obtain equivalent posterior inferences from the two models. Am I missing something here? thanks.

Answer 1

The models presented in the question are not equivalent. In Model 1 the $x_i$s are conditionally independent given $\mu, \sigma^2$. Meanwhile in Model 2 the conditional independence does not hold: Since $x_1$ is informative about $\lambda$, we have $p(x_2 \mid x_1, \mu, \sigma^2) \neq p(x_2 \mid \mu,\sigma^2)$. In fact, Model 2 is just a Gaussian observation model with unknown mean and variance, with a strange(?) prior over the variance defined as a ratio of two gamma distributed variables ($\sigma^2$ and $\lambda$).

A gamma mixture formulation equivalent to the Student's t observation model (Model 1) is instead obtained by defining a separate $\lambda_i$ for each $x_i$: \begin{align} x_i \mid \mu, \sigma^2,\lambda & \sim N(\mu, \sigma^2 / \lambda_i), i\in\{1,\ldots,n\} \\ \lambda_i & \sim G(\nu/2, \nu/2), i\in\{1,\ldots,n\} \\ \mu &\sim N(\mu_0, \sigma_0^2), \\ \sigma^2 &\sim G(\alpha, \beta), \end{align} with such conditional independence structure that the joint factorizes as $p(\mu)\,p(\sigma^2)\,\prod_{i=1}^n p(\lambda_i)\, \prod_{i=1}^n p(x_i \mid \lambda_i, \mu, \sigma^2)$.

Relationship to multivariate t

One way to view this is that Model 2 defines the conditional distribution $(x_1,\ldots,x_n) \mid \mu, \sigma^2$ as a uncorrelated multivariate t distribution while Model 1 defines $(x_1,\ldots,x_n)\mid \mu, \sigma^2$ as iid draws from univariate t-distributions. Unlike the case for multivariate Gaussians, these joint distributions are different, see e.g. the following references:

• Michael Roth, On the multivariate t distribution, Technical report LiTH-ISY-T-3059, Automatic Control, Linköping University, https://users.isy.liu.se/en/rt/roth/student.pdf
• BM Golam Kibria and Anwar H. Joarder. "A short review of multivariate t-distribution." Journal of Statistical research 40.1 (2006): 59-72.