I am trying to figure out how regression residuals are calculated using the specific example in the attached graphic.

Would I simply B-A (Red letters in graphic) to get C so: 22-30 = - 8 in this case? Would I do this for all data points and add the + and - values to get a residual value?

Additionally, for D, if I had another data set would I compute the residual for all data points for 2 predictors and the line of best fit?


Source: http://www.bisolutions.us/A-Brief-Introduction-to-Spatial-Regression.php

  • 1
    $\begingroup$ When you add up the residuals you will get $0$ (up to floating point error). With multiple predictors there is no "line" of best fit: you are fitting an affine space (a surface with two predictors, a hypersurface with more predictors). $\endgroup$ – whuber Oct 16 '14 at 16:16
  • $\begingroup$ Sorry whuber, can you dumb down the language for me? I do not understand: floating point error? affine space? hypersurface? I am left with more questions. $\endgroup$ – I Heart Beats Oct 16 '14 at 16:18
  • $\begingroup$ These things are easy to learn about on Wikipedia. $\endgroup$ – whuber Oct 16 '14 at 16:30
  • $\begingroup$ I am looking for a from first principles answer to my question. How do I use equations to calculate the residuals. That is not on wiki hence the "how to" website I was on in the first place. $\endgroup$ – I Heart Beats Oct 16 '14 at 16:42
  • 1
    $\begingroup$ I was not answering your question. Comments are for requesting clarification or for posting relevant but tangential observations. To help keep you from going astray, I have noted that (1) you accomplish nothing by summing the residuals, as suggested in the question; and (2) you might be misled by the characterization of the fit as a "line" when there is more than one predictor, so be careful. I referred you to Wikipedia for information about floating point error and hypersurfaces because the 500 characters available for comments are too few to explain these. $\endgroup$ – whuber Oct 16 '14 at 17:01

Yes, the residuals might be both positive and negative. The linear regression typically minimizes the square of them.

In case of two-dimensional input, we obtain a regression plane and the residuals are calculated in the same way.

EDIT: The regression plane is defined as $$ z_i =\beta_0+\beta_1x_{i} +\beta_2y_{i}+\epsilon_i $$ and the residual is for given parameters $\beta_0,\beta_1,\beta_2$ and given data record $(z_i,y_i,x_i)$ calculated as $$ \epsilon_i=z_i -(\beta_0+\beta_1x_{i} +\beta_2y_{i}) $$ Similarly also with higher dimensions.

  • $\begingroup$ I am confused. Is there an actual equation to calculate the residuals when a regression plane is used? Thank you. $\endgroup$ – I Heart Beats Oct 16 '14 at 16:17
  • 2
    $\begingroup$ (+1) A warning for those familiar with regression who might be reading this: the material referenced in the question uses betas (without hats) to refer to the estimated coefficients rather than the model parameters and $\epsilon_i$ to refer to the residuals rather than the random components of the model. In that sense this answer is perfectly correct--but we ought to look askance at the reference material for so abusing a conventional notation! $\endgroup$ – whuber Oct 16 '14 at 17:05
  • $\begingroup$ For those of us new to statistical notation, would you mind adding a link to the proper/commonly accepted etiquette? $\endgroup$ – I Heart Beats Oct 16 '14 at 17:30
  • 1
    $\begingroup$ @IHeartBeats, you might do better to ask that as a new question. Briefly, Greek letters are often used to represent the true values of the parameters, whereas adding hats (eg, $\beta$ vs $\hat\beta$) suggests the estimate of the parameter values that you got by analyzing your data. $\endgroup$ – gung Oct 16 '14 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.