Why is it customary to use mean square error rather than mean absolute error (absolute value of target - output)?
Specifically in neural networks - does it have anything to do with its convenient derivative?
MSE has several advantages over MAE, but also some disadvantages.
Just list some of them, include but not limited to:
Decomposition of MSE into Variance and Bias square is one of the most famous advantages. This property helps us to understand the logic behind error, especially MSE, while MAE has no such mathematical meaning.
MAE with absolute value calculation is not differentiable globally, while MSE can. This make it convenient to act as loss function and help algorithm to find the gradient method for optimization.
But as you may think, MSE weights a lot for the outliers than MAE and sensitive to outlier. Model by minimizing MSE may be affected by outlier substantially.