In base R, what do the unweighted residuals from weighted least squares (WLS) represent? Below I estimate ordinary least squares (OLS) and calculate the residual standard error (RSE). Then I estimate WLS and calculate the RSE with the weighted residuals. All is good. But why does the RSE from WLS using the unweighted residuals not match the RSE from OLS using the unweighted residuals?


x <- rnorm(25)

y <- 5 * x + rnorm(25)

unweighted <- lm(y ~ x)


sqrt(sum(resid(unweighted)^2) / 23)

w <- 1:25

weighted <- lm(y ~ x, weights = w)


sqrt(sum(weighted.residuals(weighted)^2) / 23)

sqrt(sum(w * resid(weighted)^2) / 23)

sqrt(sum(resid(weighted)^2) / 23)

1 Answer 1


Let's the the code following :

x_a <- cbind(x,1)
y_a <- y
coeff_a <- solve(t(x_a)%*%x_a)%*%t(x_a)%*%y_a
resid_a <- y_a-x_a%*%coeff_a

x_b <- diag(sqrt(w)) %*% x_a
y_b <- diag(sqrt(w)) %*% y_a
coeff_b <- solve(t(x_b)%*%x_b)%*%t(x_b)%*%y_b
resid_b <- y_b-x_b%*%coeff_b
resid_bp <- y_a-x_a%*%coeff_b


The first resolution is the ordinary least squares. The second the weighted least squares.

You can find residuals in the transformed model (resid_b). Throught, they aren't the true residuals. They are the residuals of the transformed model.

The true residuals of the WLS are resid_bp, when you have applied the coeff you found on the real inputs and made the difference with the real response.

  • $\begingroup$ @ArnaudFedlmann What, then, do summary(weighted)$residuals represent? I guess I don't understand "residuals of the transformed model". Could you explain that one part without vector notation please? $\endgroup$
    – InColorado
    Commented Mar 28, 2021 at 17:17
  • $\begingroup$ @InColorado putting weights to a linear model simply means you find the coefficients after transforming your data with a diagonal matrix (the weights you have put). Then you either have the residuals after this transformation, or you can take the coefficients you've found and see what the residuals is with those coeffs but using those coefs on the data as it is before the transformation. $\endgroup$ Commented Mar 29, 2021 at 9:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.