Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 4 down vote favorite

While reading a textbook on regression I encountered the following paragraph:

" The least squares estimate of a vector of linear regression coefficients ($\beta$) is

$$ \hat{\beta} = (X^{t}X)^{-1}{X^t}y $$

which, when viewed as a function of data $y$ (considering the predictors $X$ as constants), is a linear combination of the data. Using the Central Limit Theorem, it can be shown that the distribution of $\beta$ will be approximately multivariate normal if the sample size is large"

I'm definitely missing something from the text, but I don't understand how can a single $\beta$ value have a distribution? How are the multiple $\beta$ values generated to obtain the distribution referred to in the text?

share|improve this question
2  
$\beta$ is the vector of regression coefficients - does that clear up the confusion? – Macro May 17 '12 at 17:43
2  
When using the least squares approach, you assume that $\beta$ is fixed but unknown. However, $\hat{\beta}$, since it's a function of the (random) data, has a distribution. Asymptotically the distribution is a normal distribution. Non-asymptotically, an individual coefficient will be a t distribution. – Taylor May 17 '12 at 17:49
1  
It may help to observe that $H = (X^tX)^{-1}X^t$ is considered a constant matrix in the regression setting and that $y$ is the realization of a (vector valued) random variable. That bit about the CLT, though, is not quite correct: it is relying either on $H$ having a certain structure, which sometimes does not actually occur even with huge datasets, or else on $y$ itself being multivariate normal (but then there's no need to invoke the CLT). – whuber♦ May 17 '12 at 18:08
@Taylor But how do you know the distribution of B if the only thing I know is that the "sample size is large" ? – Dbr May 17 '12 at 18:23
@Taylor The individual component of the beta vactor will have a t distribution only if the error compoent in the regression model is Gaussian with 0 mean and constant variance. In the non-normal case you wouldn't necessarily known its distribution under the null hypothesis but it may still be asymptotically normal. However as whuber states the central limit theorem may not hold because it is a weighted average and we need to know that the weights don't chnage with sample size in a way that allows a few terms to dominate the sum. – Michael Chernick May 17 '12 at 22:18
show 2 more comments

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.