I think the ordinary least squares are the sum of the vertical distance between the observed data to the model line(regression). And residual is calculated by add up all vertical distance between each observed data and the model line(regression), not by sum of squares? I am still not clear about these concept. Thanks!

  • $\begingroup$ (+1) Welcome to our site! I searched it for an existing answer to this question--which is pretty fundamental--but turned up nothing suitable. All our questions on least squares seem to assume everybody knows exactly what it is or else they provide technical descriptions in the language of algebra and calculus. $\endgroup$ – whuber Sep 2 '14 at 16:58

You have the terms associated with the wrong concepts.

The residual for an observation is the difference between the observation (the y-value) and the fitted line. Some residuals are positive and some are negative.

$\hspace{2 cm}$ enter image description here

The usual regression line is fitted by minimizing the sum of squares of the residuals (hence 'least squares').

[The sum of the distances (the absolute values of the residuals) is not used in ordinary least squares -- but they are used in least absolute values regression (also called $L_1$ regression). Least squares is easier to fit, but many statistics packages will offer both.]

  • $\begingroup$ Hi, Thanks a lot for kindness answers. It is useful. $\endgroup$ – Jun Shi Sep 3 '14 at 15:25

Least square is an estimating method by minimizing the sum of squares of vertical distances. Residual, a quantitative measurement, is the vertical distance between the observed and the estimated using estimated parameters. Each observation has one residual (NO summing up, No square) and the residual could be negative or positive. We usually check the scatter plot of residuals to learn the fitting of a model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.