Mean absolute error OR root mean squared error? Why use Root Mean Squared Error (RMSE) instead of Mean Absolute Error (MAE)??  
Hi
I've been investigating the error generated in a calculation - I initially calculated the error as a Root Mean Normalised Squared Error.
Looking a little closer, I see the effects of squaring the error gives more weight to larger errors than smaller ones, skewing the error estimate towards the odd outlier. This is quite obvious in retrospect. 
So my question - in what instance would the Root Mean Squared Error be a more appropriate measure of error than the Mean Absolute Error? The latter seems more appropriate to me or am I missing something?
To illustrate this I have attached an example below:                 


*

*The scatter plot shows two variables with a good correlation, 

*the two histograms to the right chart the error between Y(observed )
and Y(predicted) using normalised RMSE (top) and MAE (bottom).

There are no significant outliers in this data and MAE gives a lower error than RMSE. Is there any rational, other than MAE being preferable, for using one measure of error over the other?        
 A: This depends on your loss function. In many circumstances it makes sense to give more weight to points further away from the mean--that is, being off by 10 is more than twice as bad as being off by 5. In such cases RMSE is a more appropriate measure of error.
If being off by ten is just twice as bad as being off by 5, then MAE is more appropriate.
In any case, it doesn't make sense to compare RMSE and MAE to each other as you do in your second-to-last sentence ("MAE gives a lower error than RMSE"). MAE will never be higher than RMSE because of the way they are calculated. They only make sense in comparison to the same measure of error: you can compare RMSE for Method 1 to RMSE for Method 2, or MAE for Method 1 to MAE for Method 2, but you can't say MAE is better than RMSE for Method 1 because it's smaller.
A: To put it in short, if there are many outliers then you may consider using Mean Absolute Error (also called the Average Absolute Deviation). RMSE is more sensitive to outliers than the MAE. But when outliers are exponentially rare (like in a bell-shaped curve), the RMSE performs very well and is generally preferred.
Both the RMSE and the MAE are ways to measure the distance between two vectors: the vector of predictions and the vector of target values.
MAE corresponds to the l1 norm or Manhattan norm while RMSE corresponds to the l2 norm or Euclidian Norm. The higher the norm index, the more it focuses on large values and neglects small ones
A: Here is another situation when you want to use (R)MSE instead of MAE: when your observations' conditional distribution is asymmetric and you want an unbiased fit. The (R)MSE is minimized by the conditional mean, the MAE by the conditional median. So if you minimize the MAE, the fit will be closer to the median and biased.
Of course, all this really depends on your loss function.
The same problem occurs if you are using the MAE or (R)MSE to evaluate predictions or forecasts. For instance, low volume sales data typically have an asymmetric distribution. If you optimize the MAE, you may be surprised to find that the MAE-optimal forecast is a flat zero forecast. 
Here is a little presentation covering this, and here is a recent invited commentary on the M4 forecasting competition where I explained this effect.
A: When prediction is less focal than parameter estimation, the Gauss-Markov theorem might be relevant:
In a linear model with spherical errors, OLS - the solution to the MSE minimization problem - is efficient in a class of linear unbiased estimators - there are (restrictive, to be sure) conditions under which "you can't do better than OLS".
I am not arguing this should justify using OLS almost all of the time, but it sure contributes to why (especially since it is a good excuse to focus so much on OLS in teaching).
A: 
RMSE is a more natural way of describing loss in Euclidean distance. Therefore if you graph it out in 3D, the loss is in a cone shape, as you can see above in green. This also applies to higher dimensions, although it's harder to visualize it.
MAE can be thought of as city-block distance. It isn't really as natural of a way to measure loss, as you can see in the graph in blue.
