2
$\begingroup$

The integral form of student's t distribution is given as follows [1]:

$p(x|\mu,\lambda,\nu)=\int_0^\infty \textrm{Normal}(x|\mu,(\lambda\eta)^{-1})\textrm{Gamma}(\eta|\nu/2,\nu/2)d\eta~~~~~~~~~~~~$ (1)

The complete data likelihood is written as follows:

$P(X,Z|\theta)=\prod_{i=0}^N\textrm{Normal}(x_i|\mu,(\lambda\eta_i)^{-1})\textrm{Gamma}(\eta_i|\nu/2,\nu/2)~~~~~~~~~~$(2)

I would like to ask why $\eta$ becomes $\eta_i$ when we're writing the likelihood. There are $N$ observations, single $\mu$, and $N$ instances of $\eta$. How can you see this in (1)? Is there a 1-to-1 relation between $x_i$ and $\eta_i$, or is it just a coincidence that there are $N$ instances of both? Lastly, can we see Gamma() here as a prior distribution over $\eta$ since when doing EM we find the posterior latent distribution $p(Z|X,\theta)$?

[1] A Derivation of the EM Updates for Finding the MaximumLikelihood Parameter Estimates of the Student’s t Distribution, Carl Scheffler, 22 September 2008.

$\endgroup$
5
  • 2
    $\begingroup$ supposedly there are iid observations from some common t distribution. If there were one only $\eta$, that would induce dependence, so to preserve independence independent $\eta_i$'s are necessary $\endgroup$ Commented Oct 11, 2020 at 21:42
  • $\begingroup$ @kjetilbhalvorsen But for each $\eta_i$ there is a different distribution, so $x_i$ variables are not identically distributed. Isn't it correct? $\endgroup$
    – groove
    Commented Oct 11, 2020 at 22:55
  • 2
    $\begingroup$ I don't see anything in your question that suggests the $\eta_i$ have different distributions. Indeed, (2) explicitly asserts they have a common Gamma distribution with parameters $(\nu/2,\nu/2).$ $\endgroup$
    – whuber
    Commented Oct 12, 2020 at 13:50
  • $\begingroup$ @whuber $\eta_i$ are identical, but for each $\eta_i$, the distribution of $x_i$ is different since the variance is scaled, is this correct? $\endgroup$
    – groove
    Commented Oct 14, 2020 at 9:19
  • 1
    $\begingroup$ The distribution of $x_1$ given $\eta_1$ is different from the distribution of $x_2$ given $\eta_2$ if $\eta_1\ne\eta_2$ but $x_1$ and $x_2$ are marginally iid. $\endgroup$
    – Xi'an
    Commented Nov 20, 2020 at 15:55

1 Answer 1

1
$\begingroup$

Since $$p(x|\mu,\lambda,\nu)=\int_0^\infty \varphi(x|\mu,(\lambda\eta)^{-1})\gamma(\eta|\nu/2,\nu/2)\,\text d\eta$$ the observed likelihood writes \begin{align}\prod_{i=1}^n p(x|\mu,\lambda,\nu)&=\prod_{i=1}^n\int_0^\infty \varphi(x_i|\mu,(\lambda\eta)^{-1})\gamma(\eta|\nu/2,\nu/2)\,\text d\eta\tag{1}\\ &=\prod_{i=1}^n\int_0^\infty \varphi(x_i|\mu,(\lambda\eta_i)^{-1})\gamma(\eta_i|\nu/2,\nu/2)\,\text d\eta_i\\ &=\int_{\mathbb R_+^n}\prod_{i=1}^n \varphi(x_i|\mu,(\lambda\eta_i)^{-1})\gamma(\eta_i|\nu/2,\nu/2)\,\text d\boldsymbol\eta \end{align} since $\eta$ is a dummy symbol in each of the $n$ integrals in (1), which can be written as $\eta_i$ for the $i$-th integral.

Thus the complete likelihood can be chosen as $$\prod_{i=1}^n \varphi(x_i|\mu,(\lambda\eta_i)^{-1})\gamma(\eta_i|\nu/2,\nu/2)$$

About the second question, it is counter-productive to see $\eta$ in a Bayesian light as a parameter with a Gamma prior since $\eta$ varies with the observation, which is issued from the marginal ($\eta$ integrated out) and not the conditional on $\eta$, as demonstrated above. An $n$ sample $(x_1,\ldots,x_n)$ comes with an $n$ latent sample $(\eta_1,\ldots,\eta_n)$. Contrary to a Bayesian sample, it is not possible to learn about $\eta_i$ from the sample.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.