2
$\begingroup$

Background

In reading a book chapter on Bayesian Linear Regression, I came across a general statement by the author that:

"The value of the intercept ($\alpha$) is frequently uninterpretable without also studying any ($\beta$) parameters. This is why we need very weak priors for intercepts, in many cases".

Question

I really am not clear what the author means as to the reason why the prior on $\alpha$ (the intercept), needs to be generally WIDER than the prior on $\beta$ (the slope)?

Could someone help me understand, in the context of a simple linear regression, that why we usually need to use a WIDER prior on on $\alpha$ (the intercept) compared to the prior on $\beta$ (the slope)?

$\endgroup$
  • $\begingroup$ Hint: regress any data you like against the (Christian) year--the small dataset at stats.stackexchange.com/questions/279918 will work fine. Then redo the regression against a standardized year or even a relative year, such as years since 2000. Take a look at what happens to the standard error of the estimate of the intercept. To understand it, plot the data on axes that include $(0,0)$. $\endgroup$ – whuber May 16 '17 at 18:39
  • 1
    $\begingroup$ Even without the important issue whuber raises (that the s.e. of the intercept depends on the origin of your x), intercept and slope parameters aren't even in the same units (one's in units of y the other is in units of y per unit of x) ... so how does "wider" even come into it? You can't compare them at all. $\endgroup$ – Glen_b May 17 '17 at 0:15
  • $\begingroup$ @Glen_b, so you disagree with that quote that I provided in my question from a book? (BTW, the book is called "statistical rethinking", pp. 98-99) $\endgroup$ – rnorouzian May 17 '17 at 0:19
  • $\begingroup$ The quote in no way contradicts what I said, and vice-versa. $\endgroup$ – Glen_b May 17 '17 at 0:21
  • $\begingroup$ @Glen_b, so we have two parameters that are in different metrics. You're saying that one of them is more variable than the other, but I'm not sure what you exactly mean by drawing my attention to the metrics here, could you please explain a bit (if you don't mind)? $\endgroup$ – rnorouzian May 17 '17 at 0:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.