In residual sum of squares, why do we need to square? I am learning regession, and I dont understand why do we need to square in residual sum of squares. whats wrong with just use residual sum to represent as the error value? What is the benefit of squaring the residual? 
 A: Squaring the residuals changes the shape of the regularization function. In particular, large errors are penalized more with the square of the error. Imagine two cases, one where you have one point with an error of 0 and another with an error of 10, versus another case where you have two points with an error of 5. The linear error function will treat both of these as having equal sum of residuals, while the squared error will penalize the case with the large error more.
With a squared residual, your solution will prefer more small errors to having any large errors. The linear residual is indifferent, not caring whether the total error is all coming from one sample or spread out as a sum of many tiny errors.
You could also raise the error to a higher power to penalize large errors even more. Summing the tenth power of the residuals, for example, would likely result in a solution that has small errors for most points, but no large errors for any one point.
A: @ocram's answer is good, but one point I'd add is the connection between least squares and maximum likelihood estimation.  If we have a regression model of the form $y_i = \beta_0 + \sum_{j=1}^{p} \beta_j x_{ij} + \epsilon_i$ where the $\epsilon_i$ are independent normal$(0, \sigma^2)$ random variables then the likelihood function becomes
$$
\mathcal{L}(\beta) = \frac{1}{\sigma^n \sqrt{2 \pi}^n} \exp \left ( - \frac{\sum_{i=1}^{n} (y_i - \beta_0 - \sum_{j=1}^{p} \beta_j x_{ij})^2}{2 \sigma^2} \right ) .
$$
If we want to maximize this as a function of $\beta$ that's equivalent to minimizing $\sum_{i=1}^{n} (y_i - \beta_0 - \sum_{j=1}^{p} \beta_j x_{ij})^2$, and this is nothing but the least squares criterion.
It's also interesting to note that means themselves are least squares estimates in the univariate case, so if we agree that means are good things to look at then least squares makes sense.
A: If you do not square, a negative residual (below the line) can offset the impact of a positive residual (above the line). Squaring is a remedy. Taking the absolute values of the residuals provides an alternative. But squaring is much easier to handle from a mathematical point of view (cf. derivatives).
