0
$\begingroup$

It is a Bayesian Regression, y and X are observed data, the response and the feature matrix respectively. \begin{eqnarray*} \mathbf y \mid \mathbf X,\mathbf w,\sigma^2 &\sim& N (\mathbf X \mathbf w, \sigma^2 \mathbf I ) \\ \mathbf w\mid \lambda &\sim &N(\mathbf 0, \lambda \mathbf I) \\ \sigma, \lambda >0 \end{eqnarray*}

How could I show that $\mathbf w \mid \mathbf X , \mathbf y , \lambda , \sigma^2$ is multivariate Gaussian? And what is the mean vector and variance-covariance matrix? I have been looking into this for many days and dont know how to solve it at all. Thanks!

$\endgroup$
1
  • 1
    $\begingroup$ Please add the self-study tag & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. $\endgroup$ Nov 24, 2021 at 10:55

1 Answer 1

0
$\begingroup$

The posterior distribution for $\textbf{w}$ will come from the product

$$p(\textbf{w}|\textbf{y},\textbf{X},\lambda,\sigma^{2}) \propto N(\textbf{X} \textbf{w},\sigma^{2}I)N(0,\lambda I)$$

by discarding all the terms that do not include $\textbf{w}$.

If you do that you will end up with

$$e^{-\frac{1}{2\sigma^{2}}(-2\textbf{w}^{T}\textbf{X}\textbf{y}+\textbf{w}^{T}\textbf{X}^{T}\textbf{X}\textbf{w})-\frac{1}{2\lambda}\textbf{w}^{T}\textbf{w}} \ \ \ \ (*)$$

Now let's assume that we have a multivariate normal distribution $\textbf{z}\sim N(\textbf{m},\Sigma)$, if we expand each pdf and keep only terms that have $\textbf{z}$, we will end up with

$$e^{-\frac{1}{2}(\textbf{z}^{T}\Sigma^{-1}\textbf{z}-2\textbf{z}\Sigma^{-1}\textbf{m})} \ \ \ \ \ (**)$$

Hence, in order to identify the mean and covariance of $\textbf{w}$ try to bring $(*)$ into the form $(**)$ and identify which terms correspond to $\Sigma^{-1}$ and $\textbf{m}$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.