2
$\begingroup$

it's well known that the scale mixture of normal distributions is equivalent to a Student t model, that is $$ t_{(v)}(x|\mu,\sigma^2)=\int_0^\infty N(x|\mu,\sigma^2/\lambda)\times G(\lambda|v/2,v/2)d\lambda. $$

a full Bayesian analysis is completed by assigning prior distributions for $\mu$ and $\sigma^2$, say $$ \mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta), $$ where $\mu_0,\sigma_0^2,\alpha,\beta, v$ known.

Considering a random sample $(x_1,...x_n)$, modeled according to the following models:

Model I: $(x_1,...x_n)|\mu,\sigma^2\sim t_{(v)}(\mu,\sigma^2)$

$\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$

Model II: $(x_1,...x_n)|\mu,\sigma^2,\lambda\sim N(\mu,\sigma^2/\lambda)$

$\lambda\sim G(\lambda|v/2,v/2)$

$\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$

When I simulate from the posterior distribution from each model, the posterior estimates for $\mu$ and $\sigma^2$ are not the same in the two models. I was expecting to obtain equivalent posterior inferences from the two models. Am I missing something here? thanks.

$\endgroup$
5
$\begingroup$

The models presented in the question are not equivalent. In Model 1 the $x_i$s are conditionally independent given $\mu, \sigma^2$. Meanwhile in Model 2 the conditional independence does not hold: Since $x_1$ is informative about $\lambda$, we have $p(x_2 \mid x_1, \mu, \sigma^2) \neq p(x_2 \mid \mu,\sigma^2)$. In fact, Model 2 is just a Gaussian observation model with unknown mean and variance, with a strange(?) prior over the variance defined as a ratio of two gamma distributed variables ($\sigma^2$ and $\lambda$).

A gamma mixture formulation equivalent to the Student's t observation model (Model 1) is instead obtained by defining a separate $\lambda_i$ for each $x_i$: \begin{align} x_i \mid \mu, \sigma^2,\lambda & \sim N(\mu, \sigma^2 / \lambda_i), i\in\{1,\ldots,n\} \\ \lambda_i & \sim G(\nu/2, \nu/2), i\in\{1,\ldots,n\} \\ \mu &\sim N(\mu_0, \sigma_0^2), \\ \sigma^2 &\sim G(\alpha, \beta), \end{align} with such conditional independence structure that the joint factorizes as $p(\mu)\,p(\sigma^2)\,\prod_{i=1}^n p(\lambda_i)\, \prod_{i=1}^n p(x_i \mid \lambda_i, \mu, \sigma^2)$.

Relationship to multivariate t

One way to view this is that Model 2 defines the conditional distribution $(x_1,\ldots,x_n) \mid \mu, \sigma^2$ as a uncorrelated multivariate t distribution while Model 1 defines $(x_1,\ldots,x_n)\mid \mu, \sigma^2$ as iid draws from univariate t-distributions. Unlike the case for multivariate Gaussians, these joint distributions are different, see e.g. the following references:

  • Michael Roth, On the multivariate t distribution, Technical report LiTH-ISY-T-3059, Automatic Control, Linköping University, https://users.isy.liu.se/en/rt/roth/student.pdf
  • BM Golam Kibria and Anwar H. Joarder. "A short review of multivariate t-distribution." Journal of Statistical research 40.1 (2006): 59-72.
$\endgroup$
  • $\begingroup$ Thanks for your reply. Considering the model you pointed as equivalent to Model I, should I integrate over all the $\lambda_i $ in order to prove the equivalence? Thanks. $\endgroup$ – Ludwig May 12 '18 at 0:25
  • $\begingroup$ @Ludwig in principle yes, but note that due to the factorization the integral factorizes into integrals of $p(\lambda_i)\,p(x_i \mid \lambda_i, \mu, \sigma^2) d \lambda_i$ $\endgroup$ – Juho Kokkala May 22 '18 at 6:56
  • $\begingroup$ Juho,considering Model II: $(x_1,...x_n)|\mu,\sigma^2,\lambda\sim N(\mu,\sigma^2/\lambda)$, $\lambda\sim G(\lambda|v/2,v/2)$, $\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$. Now Model III: $$ \mathbf{x}=(x_1,x_2,...,x_n)|\mu,\sigma^2 \sim MVT_{(v+n)}(\mathbf{x}|\mathbf{\mu},\sigma^2I), $$ where $\mathbf{\mu}=(\mu,...,\mu)'$,MVT stands for "multivariate t distribution", $\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$ Are models II and III equivalent as far as the marginal posterior distributions of $\mu$ and $\sigma^2$ are concerned? $\endgroup$ – Ludwig Nov 13 '18 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.