Getting started with analysis of residuals in linear regression

Question

Since the mean of the residuals should be close to zero and with my calculations yield the following result:

> mean(resid(trees.lm)
[1] -3.065293e-17

is it correct to stipulate that the mean is close to zero?

My second question is as follows: While I am working with the gamble data set in my class, I need to compute the correlation of the residuals with income. How would I set up the calculation? I'm thinking of cor(residuals(data.lm)).

Answer 1

The answer is a very definite no.

The reason is that the size of something is almost always relative: when you ask whether the mean residual is "small," you have to have some sense of small compared to what.

For example, consider these data:

set.seed(17)
x <- (-5):5 / 5
y <- 10^(-8)*(x + rnorm(length(x)))
fit <- lm(y ~ x)
mean(residuals(fit))

The output is 3.763354e-26, which is one billionth the size of your mean residual. By comparison, your mean residual is enormous. That's why you have no right to claim your mean residual is small, at least not without further investigation.

A reasonable measure of size for your comparison is the typical size of a residual:

mean(abs(residuals(fit))

For these sample data, the mean absolute residual is 4.942933e-09. Compared to this, the mean residual is only 3.763354e-26 / 4.942933e-09 = about $10^{-17}$ as big. That's like comparing the diameter of a proton to the size of a city; it's a tiny number indeed. (If you're not yet comfortable with the "e", or scientific, notation, you can find some introductions online, such as at http://www.nyu.edu/pages/mathmol/textbook/scinot.html.)

Do a similar calculation for your own data to find out whether the mean residual is small.

---

Your second question is about using R. Those questions are better suited for the StackOverflow site.

Answer 2

Answer to the first part. -3.065293e-17 is R-speak for 0. I am not sure what model you have fit, but if it's a basic regression, the residuals are forced, in theory, to sum to zero. But in the world of floating point arithmetic, which is what R is doing, you don't always get 0 ... rather, something very teensy.

Second part. I am not sure that I understand your question. cor() takes 2 vectors, as in cor(x,y). You need to give it your residuals and your income.

Answer 3

Quick answer for the first part: yes. Getting a more accurate answer would require some sort of idea of the dispersion of the sample mean, however.

Quick answer for part two: what do you mean by set up? Looks like you got the right idea with the code. You might be missing a parentheses, though. Keep in mind stats::cor() handles a few types of correlation.

If you're looking for the mathematical formula, I'm sure you'd get quite a bit of results with the query "Pearson Correlation."