0
$\begingroup$

Disclaimer: This question is not about the comparison between OLS, sum of squared residuals (mean based approach) and the sum of absolute residuals (median based approach), which has been already fully debated on this site. The question is about the feasibility of using a simple sum of residuals to find the best fitting line of a dataset.

Intuition: Given a set of values, its mean not only minimizes the squared residuals from the mean, but also ensures that the total sum of residuals is zero. Therefore the mean can be found leveraging either of those properties.

My Question: When performing a linear regression, wouldn't we obtain the same result of OLS if we required the total sum of residuals to simply converge to zero? In that way instead of looking for the minimum of a quadratic equation, we would directly solve a simpler linear problem.

I understand the benefit of being able to compare the goodness of fit across datasets by having the total sum of squares, but I have the feeling (and here I would like you to prove me wrong) that the problem of finding the best fitting line doesn't require a quadratic equation.

$\endgroup$
  • 2
    $\begingroup$ There's many possible coefficients $\beta$ that would produce residuals that are orthogonal to the vector of ones. Indeed, you're just specifying one equation in the normal equations. $\endgroup$ – user795305 Jul 10 '17 at 18:13
  • $\begingroup$ Have you considered the effect of a outlier with a large error - when the error is squared, the effect of the outlier on the fit is larger than if the error is not squared. $\endgroup$ – James Phillips Jul 10 '17 at 19:29
1
$\begingroup$

If I understand correctly, the OP proposes the following: instead of finding $\hat \beta$ through

$$\hat \beta:\,\,\, (\mathbf y - \mathbf X\hat \beta)'(\mathbf y - \mathbf X\hat \beta)=\min \tag{1}$$

which is a multivariable, single-valued function to be minimized over the uklnown coefficients,

we could attempt instead to calculate it through ($\mathbf 1$ is a series of ones)

$$\hat \beta:\,\,\, \mathbf 1'(\mathbf y - \mathbf X\hat \beta)=0 \tag{1}$$

which is a single equation, with many unknowns, and breaks down to

$$\hat \beta:\,\,\, \sum y_i = n\hat \beta_0 + \hat\beta_1\sum x_{1i} +...+ \hat\beta_k\sum x_{Ki}$$

$$\implies \hat \beta:\,\,\, \bar y = \hat\beta_0 + \hat\beta_1\bar x_1 +...+ \hat\beta_k\bar x_K$$

Well, a solution to this last one could be

$$\hat \beta = (\hat \beta_0 = \bar y, \hat \beta_1=0,\hat \beta_2=0,etc,...,)$$

Another could be

$$\hat \beta = (\hat \beta_0 = 0, \hat \beta_1 = \bar y/\bar x_1,\hat \beta_2=0,etc...,)$$

etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.