8
$\begingroup$

I am running linear regression for a continuous variable (not standardized) with age and 2 other numeric continuous variables (not standardized), 2 categorical variables with 3 levels each and 1 categorical variable with 2 levels (gender). Total number of cases (rows) is about 12k.

I am getting P values for all variables to be highly signficant and adjusted R-squared is 0.618.

I am getting following QQ plot with standardized residuals:

enter image description here

What is the diagnosis? What does this shape of QQ plot indicate? Also what should I do (if any) to improve my model?

The residual vs fitted plot is as follows:

enter image description here

Edit: My question is different from How to interpret a QQ plot since I am asking details about this particular shape of residual QQ plot, not about all shapes.

Edit2: In response to answer by @gung, the boxplot of residuals for categorical variables (CatVar 1,2 and 3) is shown below:

enter image description here

$\endgroup$
7
  • 3
    $\begingroup$ Does this answer your question? How to interpret a QQ plot $\endgroup$ – COOLSerdash Aug 4 '20 at 7:03
  • 3
    $\begingroup$ The Q-Q-plot seems to indicate that the residuals are heavier tailed compared to the normal distribution. $\endgroup$ – COOLSerdash Aug 4 '20 at 7:04
  • $\begingroup$ How do I change regression to improve it? $\endgroup$ – rnso Aug 4 '20 at 9:35
  • 1
    $\begingroup$ To be totally honest, with a sample size of 12k, I wouldn't worry too much about the deviations from normality. Residual normality is often stated as one of the weaker assumptions (see Andrew Gelman, for example). Residual normality and homoscedasticity are more important when you're using the model to make predictions. $\endgroup$ – COOLSerdash Aug 4 '20 at 9:54
  • 1
    $\begingroup$ It's rare that a single diagnostic plot tells all that you might learn about the fit of a model. In interpreting it you need to bring to bear subject-matter expertise and consideration of what else is in your model. Mentions of age and gender suggest social science and/or medical science as the area. I would want to look at other plots such as residual versus fitted. For example, it's possible that you have some kind of heterogeneity. Do the points in either tail have something else in common? Or do you just have error distributions heavier than the normal? $\endgroup$ – Nick Cox Aug 4 '20 at 13:51
3
$\begingroup$

The set of examples in How to interpret a QQ plot includes the basic shape in your question. Namely, the ends of the line of points turn counter-clockwise relative to the middle. Given that sample quantiles (i.e., your data) are on the y-axis, and theoretical quantiles from a standard normal are on the x-axis, that means the tails of your distribution are fatter than what you would see from a true normal. In other words, those points are much further from the mean than you would expect if the data generating process were actually a normal distribution.

There are lots of distributions that are symmetrical and have fatter tails than the normal. I would often start by looking at $t$-distributions, because they are well understood, and you can adjust the tail 'fatness' by modulating the degrees of freedom parameter. Your example is notable in that the middle is very straight, and the ends are also very straight and roughly parallel to each other, with fairly sharp corners in between. That suggests you have a mixture of two distributions with the same mean, but different standard deviations. I can generate a plot that looks pretty similar to yours pretty easily in R with the following code:

set.seed(646)                      # this makes the example exactly reproducible
s = 4                              # this is the ratio of SDs
x = c(rnorm(11600, mean=0, sd=1),  # 99.7% of the data come from the 1st distribution
      rnorm(  400, mean=0, sd=s))  # small fraction comes from 2nd dist w/ greater SD
qqnorm(x)                          # a basic qq-plot

enter image description here

A better way to determine the mixing proportions and relative SDs would be to fit a Gaussian mixture model. In R, that can be done with the Mclust package, although any decent statistical software should be able to do it. I demonstrate a basic analysis in my answer to How to test if my distribution is multimodal?

You might also simply make some boxplots of your residuals as a function of your categorical variables, either individually or in specified combinations. It may well be that the heteroscedasticity can be easily found and yield meaningful insights into your data.

As @COOLserdash noted, I wouldn't worry about this for purposes of statistical inference, although if you can identify a heterogeneous subgroup, you can model your data using weighted least squares. For purposes of prediction, mean predictions should be unaffected by this, but prediction intervals based on normality will be incorrect and yield 'black swans' and occasionally cause problems. So long as you don't collapse the global financial system, it might not be so bad. You could just make the prediction intervals wider, or you could again model it, especially if the subgroups are identifiable.

$\endgroup$
3
  • $\begingroup$ Very well explained! Anything to analyze from residual vs fitted plot in this case? I had added that also in my question above. $\endgroup$ – rnso Aug 5 '20 at 1:45
  • $\begingroup$ I have added boxplots of residuals with categorical variables in my question above. They do not seem to show anything remarkable. $\endgroup$ – rnso Aug 5 '20 at 3:47
  • $\begingroup$ It's certainly possible that the heterogeneity isn't strictly a function of any of the manifest variables in your dataset, @rnso. Still, you can try combinations of your categorical variables. You can also try going in the other direction: Identify the 'outliers' & see how they differ from the rest of your data. Looking at the residuals vs fitted plot, I see that a lot of the 'outliers' have predicted values in the middle. W/o knowing more about your data, I can't say what that means. $\endgroup$ – gung - Reinstate Monica Aug 5 '20 at 17:08
4
$\begingroup$

A comment with QQ-plots of data from $\mathsf{T}(3)$ and $\mathsf{Laplace}(0,1)$ (Wikipedia) distributions, both with heavy tails.

Following up on @COOLSerdash's Comment, I'll show you QQ-plots of data sampled from a couple of distributions that have heavier tails than a normal distribution.

set.seed(2020)
v = rt(150, 3)     # Student's t, DF = 3
plot(qqnorm(v))
 points(qqline(v))

enter image description here

w = rexp(500)-rexp(500)  # difference of exponentials is Laplace
plot(qqnorm(w))
 points(qqline(w))

enter image description here

$\endgroup$
1
$\begingroup$

You should also draw a line using a qqline(), anyway, it will be always a straight line thus in your example it means that the distribution has a heavier tail in compare to normal distribution. You should consider a refit your model. However, if the effect is a strong and you fit the model to the big dataset you can also consider to left it, read more about this option here: https://www.biorxiv.org/content/10.1101/498931v1.abstract

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.