2
$\begingroup$

it's well known that the scale mixture of normal distributions is equivalent to a Student t model, that is $$ t_{(v)}(x|\mu,\sigma^2)=\int_0^\infty N(x|\mu,\sigma^2/\lambda)\times G(\lambda|v/2,v/2)d\lambda. $$

a full Bayesian analysis is completed by assigning prior distributions for $\mu$ and $\sigma^2$, say $$ \mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta), $$ where $\mu_0,\sigma_0^2,\alpha,\beta, v$ known.

Considering a random sample $(x_1,...x_n)$, modeled according to the following models:

Model I: $(x_1,...x_n)|\mu,\sigma^2\sim t_{(v)}(\mu,\sigma^2)$

$\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$

Model II: $(x_1,...x_n)|\mu,\sigma^2,\lambda\sim N(\mu,\sigma^2/\lambda)$

$\lambda\sim G(\lambda|v/2,v/2)$

$\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$

When I simulate from the posterior distribution from each model, the posterior estimates for $\mu$ and $\sigma^2$ are not the same in the two models. I was expecting to obtain equivalent posterior inferences from the two models. Am I missing something here? thanks.

$\endgroup$

1 Answer 1

5
$\begingroup$

The models presented in the question are not equivalent. In Model 1 the $x_i$s are conditionally independent given $\mu, \sigma^2$. Meanwhile in Model 2 the conditional independence does not hold: Since $x_1$ is informative about $\lambda$, we have $p(x_2 \mid x_1, \mu, \sigma^2) \neq p(x_2 \mid \mu,\sigma^2)$. In fact, Model 2 is just a Gaussian observation model with unknown mean and variance, with a strange(?) prior over the variance defined as a ratio of two gamma distributed variables ($\sigma^2$ and $\lambda$).

A gamma mixture formulation equivalent to the Student's t observation model (Model 1) is instead obtained by defining a separate $\lambda_i$ for each $x_i$: \begin{align} x_i \mid \mu, \sigma^2,\lambda & \sim N(\mu, \sigma^2 / \lambda_i), i\in\{1,\ldots,n\} \\ \lambda_i & \sim G(\nu/2, \nu/2), i\in\{1,\ldots,n\} \\ \mu &\sim N(\mu_0, \sigma_0^2), \\ \sigma^2 &\sim G(\alpha, \beta), \end{align} with such conditional independence structure that the joint factorizes as $p(\mu)\,p(\sigma^2)\,\prod_{i=1}^n p(\lambda_i)\, \prod_{i=1}^n p(x_i \mid \lambda_i, \mu, \sigma^2)$.

Relationship to multivariate t

One way to view this is that Model 2 defines the conditional distribution $(x_1,\ldots,x_n) \mid \mu, \sigma^2$ as a uncorrelated multivariate t distribution while Model 1 defines $(x_1,\ldots,x_n)\mid \mu, \sigma^2$ as iid draws from univariate t-distributions. Unlike the case for multivariate Gaussians, these joint distributions are different, see e.g. the following references:

  • Michael Roth, On the multivariate t distribution, Technical report LiTH-ISY-T-3059, Automatic Control, Linköping University, https://users.isy.liu.se/en/rt/roth/student.pdf
  • BM Golam Kibria and Anwar H. Joarder. "A short review of multivariate t-distribution." Journal of Statistical research 40.1 (2006): 59-72.
$\endgroup$
3
  • $\begingroup$ Thanks for your reply. Considering the model you pointed as equivalent to Model I, should I integrate over all the $\lambda_i $ in order to prove the equivalence? Thanks. $\endgroup$
    – Ludwig
    May 12, 2018 at 0:25
  • $\begingroup$ @Ludwig in principle yes, but note that due to the factorization the integral factorizes into integrals of $p(\lambda_i)\,p(x_i \mid \lambda_i, \mu, \sigma^2) d \lambda_i$ $\endgroup$ May 22, 2018 at 6:56
  • $\begingroup$ Juho,considering Model II: $(x_1,...x_n)|\mu,\sigma^2,\lambda\sim N(\mu,\sigma^2/\lambda)$, $\lambda\sim G(\lambda|v/2,v/2)$, $\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$. Now Model III: $$ \mathbf{x}=(x_1,x_2,...,x_n)|\mu,\sigma^2 \sim MVT_{(v+n)}(\mathbf{x}|\mathbf{\mu},\sigma^2I), $$ where $\mathbf{\mu}=(\mu,...,\mu)'$,MVT stands for "multivariate t distribution", $\mu\sim N(\mu_0,\sigma_0^2)\mbox{ and } \sigma^2\sim G(\alpha,\beta)$ Are models II and III equivalent as far as the marginal posterior distributions of $\mu$ and $\sigma^2$ are concerned? $\endgroup$
    – Ludwig
    Nov 13, 2018 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.