# How to determine the marginal pdf, the posterior?

How to get the marginal pdf of $p(y)$? Do you just integrate out $p({\sigma}^{2})$?

Say, the following joint distribution for $y \in {{R}^{d}}$ and ${{\sigma }^{2}}\in {{R}^{d}}$ IG: means inverse Gamma $${{\sigma }^{2}}\sim IG(\alpha ,\beta )\propto {{({{\sigma }^{2}})}^{-(\alpha +1)}}{{e}^{-\beta /{{\sigma }^{2}}}}$$

$$y|{{\sigma }^{2}}\sim N(\mu ,{{\sigma }^{2}}\Sigma )$$

where $a\in R$, $b\in R$,$\mu \in {{R}^{d}}$,$\Sigma \in {{R}^{d\times d}}$ are known parameters.

I know that $$p(y|\mu,\Sigma)\propto \frac{1}{{{\left| {{\sigma }^{2}}\Sigma \right|}^{\frac{n}{2}}}}\exp \left[ -\frac{1}{2}\sum\limits_{i=1}^{n}{{{\left( {{y}_{i}}-\mu \right)}^{T}}{{({{\sigma }^{2}}\Sigma )}^{-1}}({{y}_{i}}-\mu )} \right]$$ $$\propto \frac{1}{{{\left| {{\sigma }^{2}}\Sigma \right|}^{\frac{n}{2}}}}\exp \left[ -\frac{1}{2}\sum\limits_{i=1}^{n}{{{\left( {{y}_{i}}-\bar{y} \right)}^{T}}{{({{\sigma }^{2}}\Sigma )}^{-1}}({{y}_{i}}-\bar{y})-\frac{n}{2}{{(\bar{y}-\mu )}^{T}}{{({{\sigma }^{2}}\Sigma )}^{-1}}(\bar{y}-\mu )} \right]$$ $$\propto \frac{1}{{{\left| {{\sigma }^{2}}\Sigma \right|}^{\frac{n}{2}}}}\exp \left[ -\frac{n-1}{2}tr\left( {{({{\sigma }^{2}}\Sigma )}^{-1}}S \right)-\frac{n}{2}{{(\bar{y}-\mu )}^{T}}{{({{\sigma }^{2}}\Sigma )}^{-1}}(\bar{y}-\mu ) \right]$$

what I have done is using

$$p{({\sigma}^{2})}{\times}p(y|{{\sigma }^{2}})$$

which gives me, $$\propto {{\left( {{\sigma }^{2}} \right)}^{-(\alpha +1)}}{{e}^{-\beta /{{\sigma }^{2}}}}\times \frac{1}{{{\left| {{\sigma }^{2}}\Sigma \right|}^{\frac{n}{2}}}}\exp \left[ -\frac{n-1}{2}tr\left( {{({{\sigma }^{2}}\Sigma )}^{-1}}S \right)-\frac{n}{2}{{(\bar{y}-\mu )}^{T}}{{({{\sigma }^{2}}\Sigma )}^{-1}}(\bar{y}-\mu ) \right]$$

This doesn't look like anything to me, am I even on the right track?! Anyways, if any experts knows, please point me out, thanks sooo much!

• Is this homework? If so, please add the "homework" tag... we'll still help guide you to an answer (rather than just giving it to you.) – jbowman Feb 20 '12 at 18:24
• The answer is no! – Xi'an Feb 20 '12 at 19:11
• More seriously, your last line is completely correct. So you are on the right track: remember that only $\sigma^2$ varies in this expression and that you can factorise most of the term in the exponential in $\sigma^2$. Then you should see a standard distribution in $\sigma^2$. – Xi'an Feb 20 '12 at 21:04
• So, I think I will integrate the joint density(my last line) w.r.t. $\sigma^{2}$, then I get p(y). Is that also what you are saying? But I am not too sure about integration matrix. I don't get what you mean by "factorise them". Something like complete the square? – user1061210 Feb 20 '12 at 21:27
• Let me cook dinner and I will explain..! – Xi'an Feb 21 '12 at 19:19

What you get as your bottom line is of the form $$(\sigma^2) ^{-\alpha-1-nd/2}\exp\{-A\sigma^{-2}\}$$ so the posterior distribution in $\sigma^{-2}$ is an inverse gamma distribution. (Note that $$\text{tr}((\sigma^2\Sigma)^{-1}S)=\sigma^{-2}\text{tr}(\Sigma^{-1}S)\,.)$$ From this property, you can derive the normalising constant.

• Does this look right? $p(y)=\frac{1}{{{\left| \Sigma \right|}^{n/2}}}\frac{\Gamma (a+n/2)}{{{\left( b+\frac{n-1}{2}tr\left( {{\Sigma }^{-1}}S \right)+\frac{n}{2}{{(\bar{y}-\mu )}^{T}}{{(\Sigma )}^{-1}}(\bar{y}-\mu ) \right)}^{a+n/2}}}$ – user1061210 Feb 22 '12 at 3:31
• this is not any distribution I could recognize, more like a normalising constant with y bar in it. – user1061210 Feb 22 '12 at 3:32
• Note that you can write the denominator as $\beta+(y-\mu)^T\Sigma^{-1}(y-\mu)$. note that the degrees of freedom is equal to double the power of this term minus the dimension of $y$ which gives us $v=2\alpha+n-d$. thus we can write posterior as $$p(y)\propto\left[1+\frac{1}{v}(y-\mu)\left(\frac{\beta}{v}\Sigma\right)^{-1}(y-\mu)\right]^{-\frac{v+d}{2}}$$ this is the kernel of a student t distribution. – probabilityislogic Feb 22 '12 at 5:09
• Yes, this is a Student's $t$ density. – Xi'an Feb 22 '12 at 5:19
• Apologies the above is a bit wrong. The exponent should be $\frac{v+nd}{2}$ and $v=2\alpha+n-nd$. we should also have a block diagonal matrix with $n$ blocks equal to $\Sigma$ instead of just $\Sigma$. also the term $(y-\mu)$ should be replaced by the $1\times nd$ row vector $\left[(y_1-\mu)^T,(y_2-\mu)^T,\dots,(y_n-\mu)^T\right]$. – probabilityislogic Feb 22 '12 at 5:44

Note that the normalising constant for a IG variable is

$$\frac{b^a}{\Gamma(a)}$$

This is equal to the reciprical of the integral over $\sigma^{2}$ of the kernel of the pdf. hence we have

$$\int_0^{\infty}(\sigma^{2})^{-(a+1)}\exp\left(-\frac{b}{\sigma^2}\right)d\sigma^2=\frac{\Gamma(a)}{b^a}$$

Your integral is of this form for certain choice of $a$ and $b$.

• @user1061210 - the inverse gamma distribution is undefined for $\sigma^2\leq 0$. The integral is then infinite for negative range. You cannot have negative range for this parameter, as it is a normal distribution variance. Similarly $\Sigma$ must be positive semi-definite (i.e. $x^T\Sigmax\geq 0$ for any vector $x$) for your distribution for $p(y|\sigma^2)$ to be valid – probabilityislogic Feb 22 '12 at 2:45
• Does this look good? continued from my last line $$={{\left( {{\sigma }^{2}} \right)}^{-(\alpha +1)-n/2}}\frac{1}{{{\left| \Sigma \right|}^{n/2}}}\exp \left\{ -\frac{1}{{{\sigma }^{2}}}\left[ b-\frac{n-1}{2}tr\left( {{\Sigma }^{-1}}S \right)-\frac{n}{2}{{(\bar{y}-\mu )}^{T}}{{(\Sigma )}^{-1}}(\bar{y}-\mu ) \right] \right\}$$ – user1061210 Feb 22 '12 at 3:28
• integrate out ${sigma}^{2}$ $$=\frac{1}{{{\left| \Sigma \right|}^{n/2}}}\int_{0}^{\infty }{{{\left( {{\sigma }^{2}} \right)}^{-(a+1+n/2)}}\exp \left\{ -\frac{1}{{{\sigma }^{2}}}\left[ b+\frac{n-1}{2}tr\left( {{\Sigma }^{-1}}S \right)+\frac{n}{2}{{(\bar{y}-\mu )}^{T}}{{(\Sigma )}^{-1}}(\bar{y}-\mu ) \right] \right\}}d{{\sigma }^{2}}$$ – user1061210 Feb 22 '12 at 3:30
• $p(y)=\frac{1}{{{\left| \Sigma \right|}^{n/2}}}\frac{\Gamma (a+n/2)}{{{\left( b+\frac{n-1}{2}tr\left( {{\Sigma }^{-1}}S \right)+\frac{n}{2}{{(\bar{y}-\mu )}^{T}}{{(\Sigma )}^{-1}}(\bar{y}-\mu ) \right)}^{a+n/2}}}$ – user1061210 Feb 22 '12 at 3:31
• Yep you got it. Note that this is called a multivariate student distribution. It is not properly normalised though. So you should replace $=$ with $\propto$ – probabilityislogic Feb 22 '12 at 3:51