6
$\begingroup$

Assume that you have a regression with a whole set of variables and you know that the residuals are not normal distributed. So you just estimate a regression using OLS to find the best linear fit. For this you disclaim the assumption of normal distributed error terms. After the estimation you have 2 "significant" coefficients. But how can anyone interpret these coefficients? So there is no way to say: "These coefficients are significant", although the Hypothesis $\beta=0$ can be declined with a high t-statistic (because of disclaiming normal error assumption). But what to do in this case? How would you argue?

$\endgroup$
3
$\begingroup$

If the residuals are not normal (and note that this applies to the theoretical residuals rather than the observed residuals), but not overly skewed or with outliers then the Central Limit Theorem applies and the inference on the slopes (t-tests, confidence intervals) will be approximately correct. The quality of the approximation depends on the sample size and the degree and type of non-normality in the residuals.

The CLT works fine for the inference on the slopes, but does not apply to prediction intervals for new data.

If your not happy with the CLT argument (small sample sizes, skewness, just not sure, want a second opinion, want to convince a skeptic, etc.) then you can use bootstrap or permutation methods which do not depend on the normality assumption.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ @Gregg You and @mpiktas seem to be saying similar things, so I believe comments about his reply may also pertain to yours. $\endgroup$ – whuber Jun 11 '11 at 17:52
1
$\begingroup$

If the errors are not normally distributed, asymptotic results can be used. Suppose your model is

$$y_i=x_i'\beta+\varepsilon_i$$

where $(y_i,x_i',\varepsilon_i)$, $i=1,...,n$ is an iid sample. Assume

\begin{align*} E(\varepsilon_i|x_i)&=0 \\ E(\varepsilon_i^2|x_i)&=\sigma^2 \end{align*}

and

$$rank(Ex_ix_i')=K,$$

where $K$ is the number of coefficients. Then usual OLS estimate $\hat\beta$ is asymptoticaly normal:

$$\sqrt{n}(\hat\beta-\beta)\to N(0,\sigma^2E(x_ix_i'))$$

Practical implications of this result are that the usual t-statistics become z-statistics, i.e. their distribution is normal instead of Student. So you can interpret t-statistics as usual, only p-values should be adjusted for normal distribution.

Note that since this result is asymptotic, it does not hold for small sample sizes. Also the assumptions used can be relaxed.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I'm not clear about the use of asymptotic arguments in this case. $\endgroup$ – Frank Harrell Jun 11 '11 at 12:15
  • 1
    $\begingroup$ I second @Frank's remark and would add that the very first thing one suspects when residuals appear non-normal is that one or both of the first two expectation assumptions is wrong: that is, there may be lack of fit ($E[\varepsilon_i|x_i] \ne 0$ for some $x_i$) or heteroscedasticity ($E[\varepsilon_i^2|x_i]$ varies with $x_i$). $\endgroup$ – whuber Jun 11 '11 at 17:51
  • $\begingroup$ @Frank, @whuber, clearly I misunderstood the question, but I do not get how, could you clarify? I thought the OP asks how to interpret t-statistics if the residuals are not normal, but all other usual assumptions hold. If this is not the case, the question is too broad. $\endgroup$ – mpiktas Jun 11 '11 at 20:19
  • $\begingroup$ My comment was related to the extremely high type II error that can result from a failure of the normality assumption to hold, and also about the "how large is large" question about the sample size. t-tests may perform poorly on the regression parameters even if all other assumptions hold. $\endgroup$ – Frank Harrell Jun 11 '11 at 20:51
  • $\begingroup$ @whuber and @frank, I know I have little experience compared to yours, but this is what I think: Using OLS to estimate the $\beta$s is ok, the only problem comes is while performing inference. (am I correct?). In inference, as @greg and @mpiktas pointed out, if symmetry holds and CLT kicks in, using t-test would not be that bad (or simply using normal distribution would be ok). I would definitely perform bootstrap of residuals for inference. I would also check Box-Cox transformation for elimination of possible variations from normality. $\endgroup$ – suncoolsu Jun 12 '11 at 8:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.