Suppose my regression model is $$Y_i = \beta_0 + \beta_1X_{i1} + \epsilon_i $$ In most books I am seeing that the prior used for precision $\tau = 1/\sigma^2 $ is $Gamma(\epsilon, \epsilon)$. However I am unable to understand how is that found given that the beta's are unknown too? Similarly I am not getting how is Normal prior used for the beta's.

  • 1
    $\begingroup$ Betas could be negative. Precisions must be positive. $\endgroup$ – Nick Cox Apr 8 '15 at 14:13
  • $\begingroup$ but why exactly these distributions? $\endgroup$ – Durin Apr 8 '15 at 14:15

A simple linear regression model is $$ Y_i = \beta_0 + \beta_1 X_{i1} + \epsilon_i, \epsilon_i \stackrel{ind}{\sim} N(0,\sigma^2). $$ A frequently used prior is $$ p(\beta_0,\beta_1,\sigma^2) \propto 1/\sigma^2 $$ Notice that this is a joint prior for $\beta_0$, $\beta_1$ and $\sigma^2$. Marginally the priors for $\beta_0$ and $\beta_1$ are uniform over the real line and the marginal prior for $\sigma^2$ is the limit as $\epsilon \to 0$ of an inverse gamma distribution with shape and scale both $\epsilon$ (or equivalently a gamma distribution on the precision $\tau=1/\sigma^2$ with the shape and rate both $\epsilon$).

You question appears to be why do we use this prior and I can think of two reasons: 1) it corresponds to the prior derived by some methods of deriving default priors (I believe a particular reference prior approach and the maximal data information approach) and 2) the resulting posterior exactly matches a non-Bayesian solution, i.e. inference through the sampling distributions of the MLE.

| cite | improve this answer | |
  • $\begingroup$ Thank you for correcting my linear model mistake. If the betas have uniform over the line then is the use of Normal distribution as prior just to make sure it is proper? For e.g. using a normal dist with large variance. Secondly I was looking for this proof of marginal prior for sigma^2 from the jeffry's joint prior(although it seems that is sigma^(-3)), do you have any resources to cite so that I could see the proof and try to understand it? $\endgroup$ – Durin Apr 8 '15 at 15:50
  • $\begingroup$ I am considering Jeffrey's prior for precision, which is proportional to 1/tau. I tried to take limiting distribution of gamma distribution. So for as alpha=beta and alpha->0, the part where I get stuck is lim(alpha->0) 1/gamma(alpha). The remaining part is 1/x. But it seems gamma function tends to positive infinity as alpha tends to 0+. Any suggestions to proceed further? $\endgroup$ – Durin Apr 10 '15 at 7:36
  • $\begingroup$ However even it tends to infinity as alpha approaches 0+, it is still a constant. So I can anyway say that distribution is proportional to 1/x. And prove what you had written in the answer. Is this approach correct? $\endgroup$ – Durin Apr 10 '15 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.