Return to Answer

We want to have a solution that minimizes the difference between the predicted and actual values.

We assume that the $y=bx+a$ i.e. there is a linear relationship.

We don't care whether the difference between predicted and actual $y$ is positive or negative and assume that distribution of errors of $y$ posses certain properties.

It then turns out that there is an analytical solution to this minimization problem. The the sum of squares of differences is the best value to minimize for a best fit.

There isn't much more to it really.

The geometrical interpretation comes handy because sum of squares has the interpretation in the form of sum of distances of the dots on the scatter plot from the $y=bx+a$ line. And human eye is very good at approximating the line that corresponds to the best fit. So it was handy before we had computers to find the fit quickly.

Nowadays it is left more as a comprehension help but is not necessary to have to understand linear regression really.

EDIT:replaced the normality of errors assumption with a correct but less concise list. Normality can be assumed for many practical cases and in that case sum of squares is optimal not only for the linear estimator and maximizes likelihood as well.

If further the assumption of normality of error distribution holds then the Sum of Squares is optimal among both linear and non-linear estimators and is maximizing the likelihood.

We want to have a solution that minimizes the difference between the predicted and actual values.

We assume that the $y=bx+a$ i.e. there is a linear relationship.

We don't care whether the difference between predicted and actual $y$ is positive or negative assume that distribution of errors of $y$ posses certain properties.

If we assume that the distribution of errors is normally distributed it turns out that there is an analytical solution to this minimization problem. The the sum of squares of differences is the best value to minimize for a best fit. But normality is not required in general case.

There isn't much more to it really.

The geometrical interpretation comes handy because sum of squares has the interpretation in the form of sum of distances of the dots on the scatter plot from the $y=bx+a$ line. And human eye is very good at approximating the line that corresponds to the best fit. So it was handy before we had computers to find the fit quickly.

Nowadays it is left more as a comprehension help but is not necessary to have to understand linear regression really.

EDIT:replaced the normality of errors assumption with a correct but less concise list. Normality was required to have an analytical solution and can be assumed for many practical cases and in that case sum of squares is optimal not only for the linear estimator and maximizes likelihood as well.

If further the assumption of normality of error distribution holds then the Sum of Squares is optimal among both linear and non-linear estimators and is maximizing the likelihood.

We want to have a solution that minimizes the difference between the predicted and actual values. We don't care whether the difference is positive or negative and assume that errors of $y$ are normally distributed. In that case the sum of squares of differences is the best value to minimize for a best fit.

If we further assume that the $y=bx+a$ then it turns out that there is an analytical solution to this minimization problem.

There isn't much more to it really.

The geometrical interpretation comes handy because sum of squares has the interpretation in the form of sum of distances of the dots on the scatter plot from the $y=bx+a$ line. And human eye is very good at approximating the line that corresponds to the best fit. So it was handy before we had computers to find the fit quickly.

Nowadays it is left more as a comprehension help but is not necessary to have to understand linear regression really.

EDIT:removed the sub-note as it is discussed in the comments.

We want to have a solution that minimizes the difference between the predicted and actual values.

We assume that the $y=bx+a$ i.e. there is a linear relationship.

We don't care whether the difference between predicted and actual $y$ is positive or negative and assume that distribution of errors of $y$ posses certain properties.

It then turns out that there is an analytical solution to this minimization problem. The the sum of squares of differences is the best value to minimize for a best fit.

There isn't much more to it really.

The geometrical interpretation comes handy because sum of squares has the interpretation in the form of sum of distances of the dots on the scatter plot from the $y=bx+a$ line. And human eye is very good at approximating the line that corresponds to the best fit. So it was handy before we had computers to find the fit quickly.

Nowadays it is left more as a comprehension help but is not necessary to have to understand linear regression really.

EDIT:replaced the normality of errors assumption with a correct but less concise list. Normality can be assumed for many practical cases and in that case sum of squares is optimal not only for the linear estimator and maximizes likelihood as well.

If further the assumption of normality of error distribution holds then the Sum of Squares is optimal among both linear and non-linear estimators and is maximizing the likelihood.

We want to have a solution that minimizes the difference between the predicted and actual values. We don't care whether the difference is positive or negative and assume that errors of $y$ are normally distributed. In that case the sum of squares of differences is the best value to minimize for a best fit.

If we further assume that the $y=bx+a$ then it turns out that there is an analytical solution to this minimization problem.

There isn't much more to it really.

The geometrical interpretation comes handy because sum of squares has the interpretation in the form of sum of distances of the dots on the scatter plot from the $y=bx+a$ line. And human eye is very good at approximating the line that corresponds to the best fit. So it was handy before we had computers to find the fit quickly.

Nowadays it is left more as a comprehension help but is not necessary to have to understand linear regression really.

Sub note: In the case of normal distribution the weight of the error should grow proportionally to it's size as the derivative of exponent is the exponent itself (see the formula of normal distribution). Thus sum of squares of errors and not just an absolute value of the difference.

We want to have a solution that minimizes the difference between the predicted and actual values. We don't care whether the difference is positive or negative and assume that errors of $y$ are normally distributed. In that case the sum of squares of differences is the best value to minimize for a best fit.

If we further assume that the $y=bx+a$ then it turns out that there is an analytical solution to this minimization problem.

There isn't much more to it really.

The geometrical interpretation comes handy because sum of squares has the interpretation in the form of sum of distances of the dots on the scatter plot from the $y=bx+a$ line. And human eye is very good at approximating the line that corresponds to the best fit. So it was handy before we had computers to find the fit quickly.

Nowadays it is left more as a comprehension help but is not necessary to have to understand linear regression really.

EDIT:removed the sub-note as it is discussed in the comments.