0
$\begingroup$

Trying to derive the posterior distribution for the following model with $n$ observations $$y=\beta_{0} + \beta_{1}X_{1}+\beta_{2}X_{2}+\epsilon$$

where the error terms $\epsilon$ follow a normal distribution $N(0,\sigma^2_{\epsilon})$ with a normal prior on $\beta_0 \sim N(0, \sigma^2_{\beta_0})$ and normal priors on coefficient terms $\beta_1, \beta_2 \sim N(0, \sigma^2_{\beta})$.

$\sigma^2_{\epsilon}$ has an inverse gamma prior with parameters $\alpha, \gamma$.

$\sigma^2_{\beta_0} \sigma^2_{\beta} \alpha, \gamma$ are constant.

Im having particular trouble defining the likelihood. Any help much appreciated!

Improve this question
$\endgroup$
6
  • $\begingroup$ Start by writing the likelihood for the simpler model $y=\beta_0+\epsilon.$ That should be (relatively) easy. Can you generalize to the model $y = \beta_0 + \beta_1 X_1 + \epsilon$? The direction should be clear; and if so, taking it to two or more explanatory variables should also be obvious. Where along this path do you run into difficulties? $\endgroup$
    – whuber ♦
    Nov 29, 2021 at 19:53
  • $\begingroup$ Thanks. My attempts teetered around writing the likelihood as a normal distribution with parameters $N(\beta_0+\beta_1X_1+\beta_2X_2,\sigma^2_{\epsilon})$. Hence we get the expression for the likelihood of $(2\pi\sigma^2_{\epsilon})^{-\frac{n}{2}} \prod_{i=1}^n \exp{(-\frac{1}{2\sigma^2_{\epsilon}}(y_i-(\beta_0+\beta_1X_1+\beta_2X_2))^2)}$ However Im not really standing on firm ground here and Im not sure how to proceed to simplify above likelihood expression so as to be able to take the product of the priors so as to eventually obtain full conditionals for each parameter. $\endgroup$
    – esc1234
    Nov 29, 2021 at 21:41
  • $\begingroup$ When you take the logarithm of the likelihood, you get a sum of quadratic functions, which will simplify nicely to a quadratic function. $\endgroup$
    – whuber ♦
    Nov 29, 2021 at 21:46
  • $\begingroup$ Thanks again, this is a big help! However taking the log results in a quadratic $(-\frac{n}{2}\log(2\pi\sigma^2_{\epsilon}) -\frac{n}{2\sigma^2_{\epsilon}}\sum_{i=1}^n (y_i-(\beta_0+\beta_1X_1+\beta_2X_2))^2$ whereby if expanded turns into an expression Im not sure how to simplify further. Can't see how to obtain the quadratic function? although Im going to continue banging my head until I get it! $\endgroup$
    – esc1234
    Nov 30, 2021 at 8:14
  • 2
    $\begingroup$ Working on it today I actually solved the problem using the likelihood I quoted above. I will write an answer for this question shortly. However I would really like to be able to solve this problem the way you are suggesting using matrix notation. Unfortunately with exams I don't have time to work on it now but I will in the new year. So I may reply to this thread then to write an alternative solution. If you would have time to give a reply then I would greatly appreciate it. Thanks again for your help! This website and others like it represents everything good about the internet! $\endgroup$
    – esc1234
    Nov 30, 2021 at 19:19

1 Answer 1

Reset to default
-1
$\begingroup$

The posterior distribution for the above linear regression model can be obtained using the likelihood

$$(2\pi\sigma^2_{\epsilon})^{-\frac{n}{2}} \prod_{i=1}^n \exp(-\frac{1}{2\sigma^2_{\epsilon}}(y_i-\beta_0-\beta_1X_1-\beta_2X_2)^2)$$

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ Can you expand on this answer to show how you arrived at it? As it stands, it's a bit sparse. $\endgroup$
    – Sycorax ♦
    Nov 30, 2021 at 21:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.