I read this sentence about the sum of squared error SSE: "The square ensures the error is always positive and larger errors are penalized more than smaller errors". I want to understand what is meant by "larger errors are penalized more than smaller errors", and how to prove that?

  • 3
    $\begingroup$ If you square small and bigger values, then the latter will change more then the first one. Plot $x$ vs $x^2$ to convince yourself. $\endgroup$
    – Tim
    Aug 16 '17 at 5:38
  • $\begingroup$ Consider accepting @Tim answer. If the error is $1$ its square is still $1$, but if the error is $100$ its square is $10000$ which is significantly larger. $\endgroup$
    – Gnattuha
    Aug 20 '17 at 19:08
  • 1
    $\begingroup$ Let an error be $\epsilon$, so that its size is $|\epsilon|$. Then its square can be factored into two terms, $$\epsilon^2 = \omega \times |\epsilon|,$$ with $\omega = |\epsilon|$. This shows how the square of any number can be considered to be its size multiplied by a weight $\omega \ge 0$. Notice that these weights are directly proportional to the sizes of the errors. That's all there is to the quotation--it is a mathematical triviality, merely involving a reinterpretation of the square of a number. $\endgroup$
    – whuber
    Aug 20 '17 at 21:05
  • $\begingroup$ "The sum of squared error" is not a question. You have to actually ask a question. $\endgroup$
    – smci
    Aug 21 '17 at 0:17

Plot $x$ against $x^2$. Smaller values squared change less, then larger values squared. The steepness of the slope of $x^2$ increases as $|x|$ grows. So if you square small errors, the penalty is smaller, then if you squared large errors. There is nothing to prove in here, it is just how the square function works.

x vs x^2


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.