2
$\begingroup$

Following this R instruction

> fit <- lm(spending ~ sex + status + income + verbal, data=spending)

I would like to calculate the mean and median of the residuals. Both my friend and I get different answers for the mean for the same data.

> mean(resid(fit))
[1] -3.065293e-17

while with the other model, results are:

> fit1<- lm(spending ~ status + income + sex + verbal, data=spending) 
> mean(resid(fit1))
[1] 4.064605e-16

Why did we get the same median but different mean if we are using the same data set?

$\endgroup$
13
  • 1
    $\begingroup$ The mean of the residuals should be zero. Small deviations from zero may be due to rounding issues. Note that your obtained values are indeed very small. $\endgroup$ – Sven Hohenstein Sep 19 '12 at 19:45
  • 1
    $\begingroup$ It's an intriguing question. (If "rounding issues" affect your friend, they ought to affect you the same way.) What have you done to rule out simple explanations, such as any difference between your data set and your friend's? A single typographical error in one of them would explain this difference. $\endgroup$ – whuber Sep 19 '12 at 19:45
  • 1
    $\begingroup$ "Small" compared to what, @Sven? See stats.stackexchange.com/questions/37522 for an account of the issues. (You and I know that mean residuals for a linear model with intercept should be exactly zero, but nevertheless an order of magnitude difference between two computations could flag something serious going on and is worth looking into: I applaud the questioner for paying attention to this.) $\endgroup$ – whuber Sep 19 '12 at 19:48
  • 1
    $\begingroup$ @Gavin Try 4.064605e-16 / -3.065923e-17! (More important point: just because all.equal uses a default tolerance of .Machine$double.eps^0.5, which is around $10^{-8}$, does not mean it is appropriate for this comparison. It's not. The appropriate tolerance depends on the typical sizes of the residuals themselves, which we do not know in this case.) $\endgroup$ – whuber Sep 19 '12 at 20:07
  • 1
    $\begingroup$ Please, register your account. That will enhance your experience with this site and will allow you to vote on and accept previous answers. $\endgroup$ – chl Sep 19 '12 at 20:12
3
$\begingroup$

By definition, the mean of the residuals of both models should be zero. Deviations are due to rounding issues.

Here's an example:

> x <- rnorm(10)
> y <- rnorm(10)
> z <- rnorm(10)
> res1 <- residuals(lm(y ~ x+z))
> res2 <- residuals(lm(y ~ z+x))

Note that both models include the same variables, but the order is different.

Look at the mean:

> mean(res1)
[1] 3.8849e-17
> mean(res2)
[1] 1.525337e-17
> median(res1)
[1] 0.06405684
> median(res2)
[1] 0.06405684

The mean is not identical, but the median is.

The residuals are nearly identical (as expected):

> all.equal(sort(res1), sort(res2))
 [1] TRUE

Check ?all.equal.

But they are not completely identical:

> identical(as.numeric(sort(res1)), as.numeric(sort(res2)))
[1] FALSE

The deviance from zero is due to rounding issues in R. Hence, the difference in the order of model predictors may cause this kind of differences.

$\endgroup$
4
  • $\begingroup$ @whuber...we are using the data from the R library and used a lm as I stated above. Could it have to do with the way we created the lm. I mean the order of predictors in our lm differ but we both used all the predictors? $\endgroup$ – MsSnowy Sep 19 '12 at 20:21
  • $\begingroup$ Yes, the difference in mean residuals is due to the order of predictors. Mathematically, both models should have mean residuals of excatly 0. $\endgroup$ – Sven Hohenstein Sep 19 '12 at 20:32
  • $\begingroup$ thank you, my friend and I were both checking if we may have omitted something in each of our linear model when we got different results. This was helpful $\endgroup$ – MsSnowy Sep 19 '12 at 21:54
  • $\begingroup$ @Gavin@whuber so technically if 5 people using the same dataset from R and the order of the variables being used are different in the lm, then each person will have different means. (Even though the data is the same) $\endgroup$ – MsSnowy Sep 20 '12 at 0:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.