5
$\begingroup$

The DHARMa package in R aims to provide scaled (quantile) residuals that, according to the DHARMa vignette,

"can be interpreted as intuitively as residuals from a linear regression"

but for generalized linear (mixed) models.

"For example, a scaled residual value of 0.5 means that half of the simulated data are higher than the observed value, and half of them lower. A value of 0.99 would mean that nearly all simulated data are lower than the observed value."

Even though this is supposed to be intuitive, I am at a loss as how to understand this concept. A linear regression residual close to zero means that the model is a good fit for the observed value. A negative residual means that the model overestimates the effect of the independent variables in that particular case. What is the equivalent of these interpretations for quantile residuals? Is it variation around 0.5?

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
3
$\begingroup$

Syre, you say about the linear regression

A linear regression residual close to zero means that the model is a good fit for the observed value. A negative residual means that the model overestimates the effect of the independent variables in that particular case.

and I think this is where the misunderstanding starts - a linear regression where you have all residuals close to zero (close by units of the standard deviation of the regression) is actually NOT a good fit. In a perfectly fitting linear regression, you assume that residuals scatter around the mean predicted value with a normal distribution. Hence, you completely expect that some values are higher and some are lower. This is not an overestimation of the effect, but a requirement of the model.

The goal of the residual checks for the linear regression is thus not to see if residuals are close to zero, but if they scatter normally distributed around zero!

The same is true for DHARMa residuals. The only difference is that the expected distribution is uniform, not normal. I quote from the vignette:

As discussed above, for a correctly specified model we would expect

  • a uniform (flat) distribution of the overall residuals

  • uniformity in y direction if we plot against any predictor.

So, interpretation of the residuals is really like in a linear regression, only that the distribution is uniform, and that the mean expectation is at 0.5.

Addition in response to the question below:

Yes, you could look at patterns in the DHARMa residuals and attempt an interpretation of why they occur, in the same way as you might do this in a linear regression.

Note that the quote in the paper assumes the most simple linear regression, where a point that is further away from the regression line is also less likely. If you include the possibility in the model that the variance of the residuals changes (e.g. in a gls), such an interpretation of raw residuals doesn't make sense any more to define outliers or especially interesting points. The most basic solution is to divide residuals by expected variance (= Pearson residuals). The quantile residuals in DHARMa generalize this idea.

A special property of the quantile residuals is that you compare against a simulated distribution. In DHARMa, I call 0 / 1 outliers, because they are outside the simulation range. What's different compared to normal outliers is that we know they are outside, but you don't know HOW FAR they are outside (you get a value of zero, if the observed value is smaller than all simulations, regardless of how much smaller). That's why this type of outliers are extra highlighted in DHARMa.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you so much for your rapid feedback! I realise how much I have yet to learn. I get your point about proximity to zero. However, my broader point, which my post reflects inadequately, is about the interpretation of residuals individually rather than as a distribution. For example, this old geography paper argues that outlying residuals can raise interesting new questions. Geography is methodologically analogous to ecology. Are quantile residuals close to 0 or 1 outliers and is it appropriate to single them out for interpretation? $\endgroup$
    – syre
    Nov 19, 2019 at 10:00
  • 1
    $\begingroup$ I've responded to this in the text $\endgroup$
    – Florian Hartig
    Nov 19, 2019 at 12:33
  • $\begingroup$ "a linear regression where you have all residuals close to zero (close by units of the standard deviation of the regression) is actually NOT a good fit" What would you say about this happening on out-of-sample data? $\endgroup$
    – Dave
    Nov 19, 2019 at 14:58
  • $\begingroup$ Dave, with the "true" = correct model, all other things equal, and if we're not overfitting, there should be no difference between performance in and out of sample. More generally, however, my point was that residual checks are not about fitting the data perfectly. Residual checks are about whether the scatter of the residuals is as one would expect from the model. $\endgroup$
    – Florian Hartig
    Nov 20, 2019 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.