Suppose we are given n data points (observations) for random variable Y and variable X. We are to find regression equation of Y on X. As I’ve read these given values of Y (observations) are realisations of random variable Y, that is, for a given X=a we have picked one value of Y=b by chance from many values of Y at that level of X.

Now suppose the regression equation after finding m and n through method of least squares comes out to be,

Y = mX + n + E , where calculated values of m and n can be substituted

My doubt is, whats the source of error? As in if this error came from the fact that Y is random and for every value of X we put in we can expect values of Y in a range

OR, the error is due to the fact that one straight line cannot determine value of Y accurately, when there are many data points. (to elaborate, suppose the purely mathematical problem of having to fit a straight line through n distinct 2-D tuples through least squares method. The error in straight line thus obtained is solely due to the fact that one straight line cannot accurately determine many distinct points.)

  • $\begingroup$ Could you explain what you mean by "error"? In many (probably most) applications $E$ isn't any kind of error or mistake at all: it simply represents deviations between a line and the data. $\endgroup$
    – whuber
    Jan 21 at 16:17

1 Answer 1


There are various potential sources of error:

  • random measurement error in Y (= unreliability in the measurement of Y)
  • prediction error/additional but omitted causes (additional variables other than X may also cause or predict Y but are not included in your model)
  • potential non-linear effects in the relationship between Y and X (like you wrote)
  • $\begingroup$ Suppose the following example. Tell me if I’m wrong.I have a height-weight dataset and I wish to find linear regression line of weight on height. Now the error could stem from the fact, that given a height, there could be many weights, but we pick one by random(which makes weight a random variable). Other source of error could be imprecise measurement of weight, say 70.3373kg for a given height. Other source of error is that a height-weight pair may not lie on regression line. My question is if this not lying on regression line is a distinct error or stems from weight being a random variable. $\endgroup$
    – Quorthon
    Jan 20 at 11:41
  • $\begingroup$ Weight may or may not be actually "random", but one thing for absolute certain is that there are many possible weights for a given height (i.e., this is nowhere near a deterministic relationship ), so one can certainly model weight as "random". Measurement error is so minor in this example that it hardly is worth mentioning. $\endgroup$ Jan 20 at 12:43
  • $\begingroup$ @BigBendRegion will a height-weight pair not lying strictly on regression line also comprise a distinct error (that is contribute to total error)or does this error stem from the fact that we don’t know the exact value of random variable? $\endgroup$
    – Quorthon
    Jan 20 at 13:12
  • 1
    $\begingroup$ It's not error, it's natural variability. Consider a group of people who all weigh 150 lb. They naturally all have different heights. $\endgroup$ Jan 20 at 14:16
  • 1
    $\begingroup$ Very hypothetical, but even accepting premise, the deterministic function could not possibly be linear. So back to real world example, deviation from linearity in indeed another source of error, but natural variability is still the largest component of variation, much larger than deviation from linearity and measurement error. $\endgroup$ Jan 21 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.