I'm trying to understand the first example in a course on fitting
models (so this may seem ludicrously simple). I've done the calculations by hand and they match the example,
but when I repeat them in R, the model coefficients are off. I thought
the difference may be due to the textbook using population variance
($\sigma^2$) whereas R may be using sample variance ($S^2$), but I
can't see where these are used in the calculations. For example, if
var() somewhere, the help section on
The denominator n - 1 is used which gives an unbiased estimator of the (co)variance for i.i.d. observations.
I have looked at the code for both
lm.fit() and neither
make use of
lm.fit() passes that data to compiled C
z <- .Call(C_Cdqrls, x, y, tol, FALSE)) which I don't have
Can anyone explain why R is giving different results? Even if there is a difference in using sample vs population variance, why do the coefficient estimates differ?
Fit a line to predict shoe size from grade in school.
# model data mod.dat <- read.table( text = 'grade shoe 1 1 2 5 4 9' , header = T); # mean mod.mu <- mean(mod.dat$shoe); # variability mod.var <- sum((mod.dat$shoe - mod.mu)^2) # model coefficients from textbook mod.m <- 8/3; mod.b <- -1; # predicted values ( 1.666667 4.333333 9.666667 ) mod.man.pred <- mod.dat$grade * mod.m + mod.b; # residuals ( -0.6666667 0.6666667 -0.6666667 ) mod.man.resid <- (mod.dat$shoe - mod.man.pred) # residual variance ( 1.333333 ) mod.man.unexpl.var <- sum(mod.man.resid^2); # r^2 ( 0.9583333 ) mod.man.expl.var <- 1 - mod.man.unexpl.var / mod.var; # but lm() gives different results: summary(lm(shoe ~ grade, data = mod.dat))
Call: lm(formula = shoe ~ grade, data = mod.dat) Residuals: 1 2 3 -0.5714 0.8571 -0.2857 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -1.0000 1.3093 -0.764 0.585 grade 2.5714 0.4949 5.196 0.121 Residual standard error: 1.069 on 1 degrees of freedom Multiple R-squared: 0.9643, Adjusted R-squared: 0.9286 F-statistic: 27 on 1 and 1 DF, p-value: 0.121
As Ben Bolker has shown, it looks like teachers make mistakes sometimes. It seems that R calculations are correct. Moral of the story: don't believe something just because a teacher says it is true. Verify it for yourself!