# Posterior mean and variance

Consider a simple linear regression model $$y=X\beta+\varepsilon$$

where $\varepsilon \sim \mathcal{N}_N(0, \sigma^2I_N)$. The priors for $\beta$ and $\sigma^2$ are split in two parts as follows: $$p(\beta,\sigma^2)=p(\beta|\sigma^2)p(\sigma^2)$$

The natural conjugate prior for $\beta$ given $\sigma^2$ is: $$p(\beta|\sigma^2) \propto |\sigma^2B|^{-1/2} \exp\Big(-\frac{1}{2}(\beta-b)'B^{-1}(\beta-b) \Big)$$

where $B$ and $\beta$ are prior parameters. For $\sigma^2$ an uninformative prior is assumed: $$p(\sigma^2) \propto \sigma^{-2}$$

I obtain the following posterior distribution: $$p(\beta,\sigma^2|y) \propto \frac{1}{\sigma^{k}} \exp\Bigg(\frac{-(\beta-b)'B^{-1}(\beta-b)}{2\sigma^2}\Bigg) \Bigg(\frac{1}{\sigma}\Bigg)^{N+2} \exp \Bigg(\frac{-(y-X\beta)'(y-X\beta)}{2\sigma^2}\Bigg)$$ Using Cholesky decomposition I arrive with the following re-arrangement: $$(\beta-b)'B^{-1}(\beta-b)=(B^{-1/2}b-B^{-1/2}\beta)'(B^{-1/2}b-B^{-1/2}\beta)$$ which allows to rewrite $$(y-X\beta)'(y-X\beta)+(B^{-1/2}b-B^{-1/2}\beta)'(B^{-1/2}b-B^{-1/2}\beta) = (w-V\beta)'(w-V\beta)$$ where $w=\binom{y}{B^{-1/2}b}$ and $V=\binom{X}{B^{-1/2}}$

Now I want to obtain the marginal posterior density of $\beta$. Without going too much through the particular steps I arrive with the following: $$p(\beta|y) \propto \Bigg(N+\frac{(\beta-\tilde{\beta})'(X'X+B^{-1})(\beta-\tilde{\beta)}}{(w-V\tilde{\beta})'(w-V\tilde{\beta})/N}\Bigg)^{-(N+k)/2}$$

where $\tilde{\beta}=(V'V)^{-1}V'w=(X'X+B^{-1})^{-1}(X'y+B^{-1}b)$

The question is how to calculate the posterior mean and variance of $\beta$. For the mean I think that I have to calculate the following integral: $$\int_{\mathbb{R}^k}\beta p(\beta|y) \text{d}\beta = \int_{\mathbb{R}^k}\beta \times C \Bigg(N+\frac{(\beta-\tilde{\beta})'(X'X+B^{-1})(\beta-\tilde{\beta)}}{(w-V\tilde{\beta})'(w-V\tilde{\beta})/N}\Bigg)^{-(N+k)/2} \text{d}\beta$$

where $C$ is the normalising constant, but I do not know how to do that. Therefore I would like to ask someone to solve this integral and show me how to obtain the variance-covariance matrix.

• You are missing a negative power in the marginal of $\beta$, which is a multivariate Student's $t$ distribution. The marginal is derived in details in our book, Bayesian Essentials with R... – Xi'an Sep 12 '17 at 7:49
• Thank you very much, I have corrected the mistake. Could you please tell me whether mean and variance of this marginal are also derived in that book? – CherryGarcia Sep 12 '17 at 8:44
• There is this on-line encyclopedia called Wikipedia that is often quite useful to get answers to this kind of questions. And yes the answer is in the book. – Xi'an Sep 12 '17 at 11:13