9
$\begingroup$

I refer to this post which seems to question the importance of the normal distribution of the residuals, arguing that this together with heteroskedasticity could potentially be avoided by using robust standard errors.

I have considered various transformations - roots, logs etc. - and all is proving useless in resolving completely the issue.

Here is a Q-Q plot of my residuals:

Normality plot

Data

  • Dependent variable: already with logarithmic transformation (fixes outlier issues and a problem with skewness in this data)
  • Independent variables: age of firm, and a number of binary variables (indicators) (Later on I have some counts, for a separate regression as independent variables)

The iqr command (Hamilton) in Stata does not determine any severe outliers which rule out normality, but the graph below suggests otherwise and so does the Shapiro-Wilk test.

Improve this question
$\endgroup$
17
  • 4
    $\begingroup$ I would not be worried by such a graph, the deviations seem mild enough. If you want you can add confidence bounds to that graph using the qenv package. $\endgroup$
    – Maarten Buis
    Jul 4, 2013 at 8:41
  • 4
    $\begingroup$ I agree with @MaartenBuis that you shouldn't worry too much based on the plot. I would not recommend to rely on a formal test of normality (e.g. Shapiro-test) of the residuals. In large samples, the test will almost always reject the hypothesis. Here is an informative answer from Glen which addresses exactly the question of formal testing of normality of residuals. $\endgroup$
    – COOLSerdash
    Jul 4, 2013 at 8:54
  • 4
    $\begingroup$ See also this and this. Note also that as your sample size gets larger, your normal assumptions become less critical. Unless you have a lot of predictors, such mild non-normality should be of no consequence at all. The problem isn't just that hypothesis tests will reject when samples are large - they answer the wrong question at other sample sizes as well. $\endgroup$
    – Glen_b
    Jul 4, 2013 at 9:17
  • 3
    $\begingroup$ The $p$-value says that the deviations from normality are larger than one would expect to happen by chance, it does not say that those deviations are large enough to endanger your model. Based on your graph, my judgement call would be that you are fine. $\endgroup$
    – Maarten Buis
    Jul 4, 2013 at 9:18
  • 5
    $\begingroup$ What matters is the effect on your inference. The only form of inference such a tiny effect would be of any impact at all would be with a prediction interval... and even there, I'd likely use it with little compunction, unless I needed a prediction interval far into the tail (say 99% or more). Of more concern would be issues like dependence and bias and mis-specification of the model for the mean or variance. $\endgroup$
    – Glen_b
    Jul 4, 2013 at 9:20

2 Answers 2

Reset to default
9
$\begingroup$

One way you can add a "test-like flavour" to your graph is to add confidence bounds around them. In Stata I would do this like so:

sysuse nlsw88, clear
gen lnw = ln(wage)

reg lnw i.race grade c.ttl_exp##c.ttl_exp union

predict resid if e(sample), resid

qenvnormal resid, mean(0) sd(`e(rmse)') overall reps(20000) gen(lb ub)

qplot resid lb ub, ms(oh none ..) c(. l l)     ///
    lc(gs10 ..) legend(off) ytitle("residual") ///
    trscale(`e(rmse)' * invnormal(@))          ///
    xtitle(Normal quantiles)

enter image description here

Improve this answer
$\endgroup$
11
  • 3
    $\begingroup$ Note that Stata users need to install qenv (by ssc install qenv) first. $\endgroup$
    – Nick Cox
    Jul 4, 2013 at 9:34
  • $\begingroup$ I'll look at this today and see if I'm able to get the confidence bounds $\endgroup$
    – Cesare Camestre
    Jul 4, 2013 at 9:35
  • $\begingroup$ Getting an error: qenvnormal resid, mean(0) se(`e(rmse)') overall reps(20000) gen(lb ub) - option se() not allowed $\endgroup$
    – Cesare Camestre
    Jul 4, 2013 at 10:05
  • 1
    $\begingroup$ correct, it should have been sd(). It is normal (no pun intended) that qenv with the overall option takes very long. $\endgroup$
    – Maarten Buis
    Jul 4, 2013 at 10:25
  • 1
    $\begingroup$ The help for qenvnormal does explain that you need to install qplot. You are expected to read the help. More importantly, I guess you are using a very old version of qplot. Install from package gr42_6 from stata-journal.com/software/sj12-1 $\endgroup$
    – Nick Cox
    Jul 4, 2013 at 10:51
5
$\begingroup$

One thing to keep in mind when examining these qq plots is that the tails will tend to deviate from the line even if the underlying distribution is truly normal and no matter how big the N is. This is implied in Maarten's answer. This is because as N gets larger and larger the tails will be farther and farther out and rarer and rarer events. There will therefore always be very little data in the tails and they will always be much more variable. If the bulk of your line is where expected and only the tails deviate then you can generally ignore them.

One way I use to help students learn how to assess their qq plots for normality is generate random samples from a distribution known to be normal and examine those samples. There are exercises where they generate samples of various sizes to see what happens as N changes and also ones where they take a real sample distribution and compare it to random samples of the same size. The TeachingDemos package of R has a test for normality that uses a similar kind of technique.

# R example - change the 1000 to whatever N you would like to examine
# run several times
y <- rnorm(1000); qqnorm(y); qqline(y)
Improve this answer
$\endgroup$
5
  • $\begingroup$ Agreed, but this was one of Maarten's key points in his answer and it's why intervals are used to signal uncertainty. $\endgroup$
    – Nick Cox
    Jul 4, 2013 at 10:53
  • $\begingroup$ Are you suggesting this answer is redundant? I think that part of this is implicit in Maarten's answer but I don't think it's a key point or complete. Maarten's answer is good. This answer is different but related. $\endgroup$
    – John
    Jul 4, 2013 at 11:10
  • $\begingroup$ It is not redundant, but a cross-reference to Maarten's answer would be likely to help future readers. $\endgroup$
    – Nick Cox
    Jul 4, 2013 at 11:14
  • $\begingroup$ To be explicit about the link between this and my answer: if you were to look under the hood of qenv you would see that this simulation technique is at the core of how the confidence bands are computed. $\endgroup$
    – Maarten Buis
    Jul 4, 2013 at 11:23
  • 1
    $\begingroup$ added a link... $\endgroup$
    – John
    Jul 4, 2013 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.