What happens if we try to minimize the sum of residuals rather than square or absolute value? Maybe a duplicate-- but I am not asking about least squares vs minimizing absolute deviations. Just simply, what if I chose to minimize the sum of residuals:
Min $\sum_i^n (Y_i-\beta_0 - \beta_1X_i)$ ?
If I try to minimize this, the second order conditions are all zero. Is there a solution to this? Would the strategy be to pick $\beta$'s so that they make the sum of residuals negative infinity?
 A: You start to reward the model for missing high, and the higher the miss, the better the loss. Indeed, you are correct to point out that the loss can be driven toward $-\infty$ as you make the intercept larger and larger.
Indeed, there is no minimum to the loss function, so slope and intercept parameters giving the minimum value do not exist.
If you then take the absolute value of the entire sum, ordinary least squares already guarantees the residuals add up to zero, so you have a minimizer in $\hat\beta_{ols}$.
A: $\sum_i^n (Y_i-\beta_0 - \beta_1X_i) = 0$ to equivalent a regression line where $\bar Y=\beta_0 + \beta_1 \bar X$ i.e. any (non-vertical) line which passes through the mean point.
One such line is the ordinary least squares result.  But all the others do so too.
Another is $\beta_1=0, \beta_0=\bar Y$, i.e. estimating a constant value for $\hat Y_i$ that ignores the $X_i$, but you can usually get a better fit than that.
You could aim to minimise the sum of the absolute deviations subject to passing through the mean point if you wished, but it would be analytically more complicated.
