10
$\begingroup$

There is a normality assumption when it comes to consider OLS models and that is that the errors be normally distributed. I have been browsing through Cross Validated and it sounds like Y and X don't have to be normal in-order for errors to be normal. My question is why when we have non-normally distributed errors is the validity of our significance statements compromised? Why will confidence intervals be too wide or narrow?

$\endgroup$
9
$\begingroup$

why when we have non-normally distributed errors is the validity of our significance statements compromised? Why will confidence intervals be too wide or narrow?

The confidence intervals are based on the way that the numerator and denominator are distributed in a t-statistic.

With normal data the numerator of a t-statistic has a normal distribution and the distribution of the square of the denominator (which is then a variance) is a particular multiple of a chi-squared distribution. When the numerator and denominator are also independent (as will only be the case with normal data, given the observations themselves are independent), the whole statistic has a t-distribution.

This then means that a t-statistic like $\frac{\hat \beta - \beta}{s_{\hat\beta}}$ will be a pivotal quantity (its distribution doesn't depend on what the true slope coefficient is, and it's a function of the unknown $\beta$), which makes it suitable for constructing confidence intervals ... and these intervals will then use $t$-quantiles in their construction to get the desired coverage.

If the data were from some other distribution, the statistic wouldn't have a t-distribution. For example, if it were heavy tailed, the t-distribution would tend to be a bit lighter tailed (the outlying observations affect the denominator more than the numerator). Here's an example. In both cases, the histogram is for 10,000 regressions:

enter image description here

The histogram on the left is for when the data are conditionally normal, n=30 (and where in this case, $\beta=0$). The distribution looks as it should. The histogram on the right is for the case when the conditional distribution is right skewed and heavy-tailed, and the histogram shows very few values outside $(-2,2)$ - the distribution isn't much like the theoretical distribution for normal data, because the statistic no longer has the t-distribution.

A 95% t-interval (which should include 95% of the slopes in our sample) runs from -2.048 to 2.048. For the normal data, it actually included 95.15% of the 10000 sample slopes. For the skewed data it includes 99.91%.

$\endgroup$
  • $\begingroup$ What distribution did you use for the skewed & heavy-tailed version? $\endgroup$ – gung Apr 23 '14 at 4:38
  • 2
    $\begingroup$ @gung Gamma with shape parameter 0.01 (sample size was 30, simple linear regression fitted); fairly similar results occur with other highly skew distributions. You don't need it nearly that skew to make the distribution look distinctly different from $t$. $\endgroup$ – Glen_b Apr 23 '14 at 6:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.