# R weighted least squares vs ordinary least squares residuals

In base R, what do the unweighted residuals from weighted least squares (WLS) represent? Below I estimate ordinary least squares (OLS) and calculate the residual standard error (RSE). Then I estimate WLS and calculate the RSE with the weighted residuals. All is good. But why does the RSE from WLS using the unweighted residuals not match the RSE from OLS using the unweighted residuals?

set.seed(1)

x <- rnorm(25)

y <- 5 * x + rnorm(25)

unweighted <- lm(y ~ x)

summary(unweighted)

sqrt(sum(resid(unweighted)^2) / 23)

w <- 1:25

weighted <- lm(y ~ x, weights = w)

summary(weighted)

sqrt(sum(weighted.residuals(weighted)^2) / 23)

sqrt(sum(w * resid(weighted)^2) / 23)

sqrt(sum(resid(weighted)^2) / 23)


Let's the the code following :

x_a <- cbind(x,1)
y_a <- y
coeff_a <- solve(t(x_a)%*%x_a)%*%t(x_a)%*%y_a
resid_a <- y_a-x_a%*%coeff_a

x_b <- diag(sqrt(w)) %*% x_a
y_b <- diag(sqrt(w)) %*% y_a
coeff_b <- solve(t(x_b)%*%x_b)%*%t(x_b)%*%y_b
resid_b <- y_b-x_b%*%coeff_b
resid_bp <- y_a-x_a%*%coeff_b

resid_b-weighted.residuals(weighted)
resid_bp-residuals(weighted)


The first resolution is the ordinary least squares. The second the weighted least squares.

You can find residuals in the transformed model (resid_b). Throught, they aren't the true residuals. They are the residuals of the transformed model.

The true residuals of the WLS are resid_bp, when you have applied the coeff you found on the real inputs and made the difference with the real response.

• @ArnaudFedlmann What, then, do summary(weighted)\$residuals represent? I guess I don't understand "residuals of the transformed model". Could you explain that one part without vector notation please? Mar 28 at 17:17
• @InColorado putting weights to a linear model simply means you find the coefficients after transforming your data with a diagonal matrix (the weights you have put). Then you either have the residuals after this transformation, or you can take the coefficients you've found and see what the residuals is with those coeffs but using those coefs on the data as it is before the transformation. Mar 29 at 9:50