3
$\begingroup$

I am trying to figure out how regression residuals are calculated using the specific example in the attached graphic.

Would I simply B-A (Red letters in graphic) to get C so: 22-30 = - 8 in this case? Would I do this for all data points and add the + and - values to get a residual value?

Additionally, for D, if I had another data set would I compute the residual for all data points for 2 predictors and the line of best fit?

Graphic

Source: http://www.bisolutions.us/A-Brief-Introduction-to-Spatial-Regression.php

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ When you add up the residuals you will get $0$ (up to floating point error). With multiple predictors there is no "line" of best fit: you are fitting an affine space (a surface with two predictors, a hypersurface with more predictors). $\endgroup$
    – whuber ♦
    Commented Oct 16, 2014 at 16:16
  • $\begingroup$ Sorry whuber, can you dumb down the language for me? I do not understand: floating point error? affine space? hypersurface? I am left with more questions. $\endgroup$
    – I Heart Beats
    Commented Oct 16, 2014 at 16:18
  • 1
    $\begingroup$ I was not answering your question. Comments are for requesting clarification or for posting relevant but tangential observations. To help keep you from going astray, I have noted that (1) you accomplish nothing by summing the residuals, as suggested in the question; and (2) you might be misled by the characterization of the fit as a "line" when there is more than one predictor, so be careful. I referred you to Wikipedia for information about floating point error and hypersurfaces because the 500 characters available for comments are too few to explain these. $\endgroup$
    – whuber ♦
    Commented Oct 16, 2014 at 17:01
  • 1
    $\begingroup$ Thank you whuber. I am looking to perform a regression from scratch. With pencil and paper and about 10 data points so that I can learn the algorithms behind it. I find these days, everything is software based, so finding equations and first principle examples is difficult. $\endgroup$
    – I Heart Beats
    Commented Oct 16, 2014 at 17:16
  • 1
    $\begingroup$ I admire that approach. $\endgroup$
    – whuber ♦
    Commented Oct 16, 2014 at 17:36

1 Answer 1

Reset to default
2
$\begingroup$

Yes, the residuals might be both positive and negative. The linear regression typically minimizes the square of them.

In case of two-dimensional input, we obtain a regression plane and the residuals are calculated in the same way.

EDIT: The regression plane is defined as $$ z_i =\beta_0+\beta_1x_{i} +\beta_2y_{i}+\epsilon_i $$ and the residual is for given parameters $\beta_0,\beta_1,\beta_2$ and given data record $(z_i,y_i,x_i)$ calculated as $$ \epsilon_i=z_i -(\beta_0+\beta_1x_{i} +\beta_2y_{i}) $$ Similarly also with higher dimensions.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I am confused. Is there an actual equation to calculate the residuals when a regression plane is used? Thank you. $\endgroup$
    – I Heart Beats
    Commented Oct 16, 2014 at 16:17
  • 2
    $\begingroup$ (+1) A warning for those familiar with regression who might be reading this: the material referenced in the question uses betas (without hats) to refer to the estimated coefficients rather than the model parameters and $\epsilon_i$ to refer to the residuals rather than the random components of the model. In that sense this answer is perfectly correct--but we ought to look askance at the reference material for so abusing a conventional notation! $\endgroup$
    – whuber ♦
    Commented Oct 16, 2014 at 17:05
  • $\begingroup$ For those of us new to statistical notation, would you mind adding a link to the proper/commonly accepted etiquette? $\endgroup$
    – I Heart Beats
    Commented Oct 16, 2014 at 17:30
  • 1
    $\begingroup$ @IHeartBeats, you might do better to ask that as a new question. Briefly, Greek letters are often used to represent the true values of the parameters, whereas adding hats (eg, $\beta$ vs $\hat\beta$) suggests the estimate of the parameter values that you got by analyzing your data. $\endgroup$
    – gung - Reinstate Monica
    Commented Oct 16, 2014 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.