0
$\begingroup$

I have a hard time to understand why a error term is included when you split a regression equation into n equations (see below). enter image description here

Lets assume this simple series. Which refers to salary and house price in four regions with data from one household in each region.enter image description here

Which will look like this if you draw a trend line.enter image description here

Here, the OLS equation will be: House price = 0 + 10 x Salary + Error term Where the error is 0 for the first and last observation and 20 for the second and third, I understand that this equation is a model of the reality, i.e., it can be used to predict the price of any household with a Salary of 10, probably the house price will differ between these households, thereof the error term.

But when it comes the example above when they divide the equation into one for each observation, I don’t understand why we need the error term.

If we consider the equations I guess it would be;

House price_1 = 0 + 10 x 10 = 100

House price_2 = 0 + 8,666 x 15 = 130 Etc.

Hope I made the case clear, hope someone can shed some light on this.

Simon

$\endgroup$
2
  • 2
    $\begingroup$ Have you noticed that the coefficients $\beta_i$ are the same in all equations? This means that each $x_i$ uniquely determines a predicted value $\beta_0+\beta_1 x_i.$ What will you do when the corresponding $y_i$ does not equal that predicted value?? $\endgroup$
    – whuber
    Nov 7, 2022 at 13:24
  • 1
    $\begingroup$ Hm, so this means that they parameters are estimated before you split the equation up? Not as in my example when I split it then estimate? $\endgroup$ Nov 7, 2022 at 13:43

1 Answer 1

0
$\begingroup$

this means that they (their?) parameters are estimated before you split the equation up?

I guess you could see the regression exercise as solving a system of $n$ equations (4 in your example) and $p$ unknowns (2 in your example). Because you have more equations than unknowns you (typically) cannot have a single solution as if you had $n = p$. So you need the error term to adjust the discrepancy between fitted value and observation in each equation. (I think before OLS, people used to select the most trustworthy datapoints or average them in order to reduce the system to $n = p$, but I can't remember a reference for this and I may be wrong...).

Also, the regression line in your example should be $y = 2.07 + 0.09x$. From:

y <- c(10, 15, 15, 20)
x <- c(100, 130, 170, 200)
fit <- lm(y ~ x)
summary(fit)

Call:
lm(formula = y ~ x)

Residuals:
      1       2       3       4 
-0.6897  1.7241 -1.7241  0.6897 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)  
(Intercept)  2.06897    3.77346   0.548   0.6385  
x            0.08621    0.02438   3.536   0.0715 .
```
$\endgroup$
1
  • $\begingroup$ Thank you, I guess my idea that the equations were totally separate from each other was what made me confused. I find it interesting that the regression line is y = 2.07 + 0.09x. I chose these observations to make it y = 0 + 0.1x since my gut told me that the Least Squares should touch (10,100) and (20,200) and pass directly between (15,130) and 15,170). $\endgroup$ Nov 8, 2022 at 9:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.