Why is it customary to use mean square error rather than mean absolute error (absolute value of target - output)?

Specifically in neural networks - does it have anything to do with its convenient derivative?

  • $\begingroup$ Probably because its because its easier to work with squares in equations than a modulus sign $\endgroup$ – Siddharth Gopi Nov 24 '13 at 1:41

MSE has several advantages over MAE, but also some disadvantages.

Just list some of them, include but not limited to:

Decomposition of MSE into Variance and Bias square is one of the most famous advantages. This property helps us to understand the logic behind error, especially MSE, while MAE has no such mathematical meaning.

MSE decomposition

MAE with absolute value calculation is not differentiable globally, while MSE can. This make it convenient to act as loss function and help algorithm to find the gradient method for optimization.

But as you may think, MSE weights a lot for the outliers than MAE and sensitive to outlier. Model by minimizing MSE may be affected by outlier substantially.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.