1
$\begingroup$

Consider a simple linear regression model \begin{equation} y=X\beta+\varepsilon \end{equation}

where $\varepsilon \sim \mathcal{N}_N(0, \sigma^2I_N)$. The priors for $\beta$ and $\sigma^2$ are split in two parts as follows: \begin{equation} p(\beta,\sigma^2)=p(\beta|\sigma^2)p(\sigma^2) \end{equation}

The natural conjugate prior for $\beta$ given $\sigma^2$ is: \begin{equation} p(\beta|\sigma^2) \propto |\sigma^2B|^{-1/2} \exp\Big(-\frac{1}{2}(\beta-b)'B^{-1}(\beta-b) \Big) \end{equation}

where $B$ and $\beta$ are prior parameters. For $\sigma^2$ an uninformative prior is assumed: \begin{equation} p(\sigma^2) \propto \sigma^{-2} \end{equation}


I obtain the following posterior distribution: \begin{equation} p(\beta,\sigma^2|y) \propto \frac{1}{\sigma^{k}} \exp\Bigg(\frac{-(\beta-b)'B^{-1}(\beta-b)}{2\sigma^2}\Bigg) \Bigg(\frac{1}{\sigma}\Bigg)^{N+2} \exp \Bigg(\frac{-(y-X\beta)'(y-X\beta)}{2\sigma^2}\Bigg) \end{equation} Using Cholesky decomposition I arrive with the following re-arrangement: \begin{equation} (\beta-b)'B^{-1}(\beta-b)=(B^{-1/2}b-B^{-1/2}\beta)'(B^{-1/2}b-B^{-1/2}\beta) \end{equation} which allows to rewrite \begin{equation} (y-X\beta)'(y-X\beta)+(B^{-1/2}b-B^{-1/2}\beta)'(B^{-1/2}b-B^{-1/2}\beta) = (w-V\beta)'(w-V\beta) \end{equation} where $w=\binom{y}{B^{-1/2}b}$ and $V=\binom{X}{B^{-1/2}}$


Now I want to obtain the marginal posterior density of $\beta$. Without going too much through the particular steps I arrive with the following: \begin{equation} p(\beta|y) \propto \Bigg(N+\frac{(\beta-\tilde{\beta})'(X'X+B^{-1})(\beta-\tilde{\beta)}}{(w-V\tilde{\beta})'(w-V\tilde{\beta})/N}\Bigg)^{-(N+k)/2} \end{equation}

where $\tilde{\beta}=(V'V)^{-1}V'w=(X'X+B^{-1})^{-1}(X'y+B^{-1}b)$


The question is how to calculate the posterior mean and variance of $\beta$. For the mean I think that I have to calculate the following integral: \begin{equation} \int_{\mathbb{R}^k}\beta p(\beta|y) \text{d}\beta = \int_{\mathbb{R}^k}\beta \times C \Bigg(N+\frac{(\beta-\tilde{\beta})'(X'X+B^{-1})(\beta-\tilde{\beta)}}{(w-V\tilde{\beta})'(w-V\tilde{\beta})/N}\Bigg)^{-(N+k)/2} \text{d}\beta \end{equation}

where $C$ is the normalising constant, but I do not know how to do that. Therefore I would like to ask someone to solve this integral and show me how to obtain the variance-covariance matrix.

$\endgroup$
  • $\begingroup$ You are missing a negative power in the marginal of $\beta$, which is a multivariate Student's $t$ distribution. The marginal is derived in details in our book, Bayesian Essentials with R... $\endgroup$ – Xi'an Sep 12 '17 at 7:49
  • $\begingroup$ Thank you very much, I have corrected the mistake. Could you please tell me whether mean and variance of this marginal are also derived in that book? $\endgroup$ – CherryGarcia Sep 12 '17 at 8:44
  • $\begingroup$ There is this on-line encyclopedia called Wikipedia that is often quite useful to get answers to this kind of questions. And yes the answer is in the book. $\endgroup$ – Xi'an Sep 12 '17 at 11:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.