2
$\begingroup$

Are the slope and intercept of a simple linear regression model always normally distributed?

Is there ever a difference between the distribution of the estimated slope and intercept and the actual ones?

I have only just begun learning about the subject but I am still not clear on the details.

A final question: is the least squares method the same as linear regression in that it gives information like the $R^2$? Thanks!

$\endgroup$
2
$\begingroup$

Are the slope and intercept of a simple linear regression model always normally distributed?

No. If the data ($y$'s) are (conditionally) normal and the other assumptions hold, they will be, and you can get asymptotic normality under some conditions, but generally, no.

Is there ever a difference between the distribution of the estimated slope and intercept and the actual ones?

Are you asking about bias? Yes, you can get bias in a variety of ways, such as errors in the $x$'s.

is the least squares method the same as linear regression in that it gives information like the $R^2$?

Least squares is the usual method (overwhelmingly so) for fitting linear regression, but you can have linear fits that don't use the usual ordinary least squares; it's still generally called 'regression'.

So the error term is what generates in some sense the distribution, is that correct? My thought was that when e is normally distributed then b0 and b1 must also be

The least squares estimates of the parameters are linear combinations of the observations. The distribution of the error (combined with the other assumptions) impacts the distribution through that.

So in multiple regression $\hat \beta = (X^\top X)^{-1} X^\top y = Ay$, say, for $A$ a $p\times n$ matrix.

Since linear combinations of multivariate normals are themselves normal, the parameter estimates are normal when the errors are.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks Glen, In something like simple linear regression then $\beta_0$ and $b_0$ have the same distribution, for example, as both take on the distribution, at every level of X, of the error term e, is that correct? And since e is normally distributed in the most basic case the slope and intercept are, since they are being defined by X and Y (e.g. when solving the minimization problems for b0 b1). Sorry if this is unclear. $\endgroup$ – user26091 May 27 '13 at 1:43
  • $\begingroup$ In simple regression $\beta_0$ doesn't have a distribution at all, it's a population parameter. I am not sure I worked out what you were trying to say. $\endgroup$ – Glen_b -Reinstate Monica May 27 '13 at 2:06
  • $\begingroup$ So the error term is what generates in some sense the distribution, is that correct? My thought was that when e is normally distributed then b0 and b1 must also be. $\endgroup$ – user26091 May 27 '13 at 2:12
  • $\begingroup$ see updated answer $\endgroup$ – Glen_b -Reinstate Monica May 27 '13 at 2:24
  • $\begingroup$ +1, this is good info & well put. However, one small question / quibble: regarding the 1st question, even if $y$ conditional on $x$ is not normally distributed, w/ sufficient $N$ the sampling distribution should be. Are we on different pages here? I wonder if I'm misinterpreting this. $\endgroup$ – gung - Reinstate Monica May 27 '13 at 3:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.