Why use a certain measure of forecast error (e.g. MAD) as opposed to another (e.g. MSE)? MAD = Mean Absolute Deviation
MSE = Mean Squared Error
I've seen suggestions from various places that MSE is used despite some undesirable qualities (e.g. http://www.stat.nus.edu.sg/~staxyc/T12.pdf, which states on p8 "It is commonly believed that MAD is a better criterion than MSE. However, mathematically MSE is more convenient than MAD.")
Is there more to it than that? Is there a paper that thoroughly analyzes the situations in which various methods of measuring forecast error are more/less appropriate? My google searches haven't revealed anything. 
A similar question to this was asked at https://stackoverflow.com/questions/13391376/how-to-decide-the-forecasting-method-from-the-me-mad-mse-sde, and the user was asked to post on stats.stackexchange.com, but I don't think they ever did.
 A: The advantages of using MAE instead of MSE are explained in Davydenko and Fildes (2016), see Section 3.1:

...Some authors (e.g., Zellner, 1986) argue that the criterion by
  which we evaluate forecasts should correspond to the criterion by
  which we optimise forecasts. In other words, if we optimise estimates
  using some given loss function, we must use the same loss function for
  empirical evaluation in order to find out which model is better.
Fitting a statistical model usually delivers forecasts optimal under
  quadratic loss. This, e.g., happens when we fit a linear regression.
  If our density forecast from statistical modelling is symmetric, then
  forecasts optimal under quadratic loss are also optimal under linear
  loss. But, if we stabilise the variance by log-transformations and
  then transform back forecasts by exponentiation, we get forecasts
  optimal only under linear loss. If we use another loss, we must first
  obtain the density forecast using a statistical model, and then adjust
  our estimate given our specific loss function (see examples of doing
  this in Goodwin, 2000).
Let’s assume we want to empirically compare two methods and find out
  which method is better in terms of a symmetric linear loss (since this
  type of loss is commonly used in modelling). If we have only one time
  series, it seems natural to use a mean absolute error (MAE). Also, MAE
  is attractive as it is simple to understand and calculate (Hyndman,
  2006)...

References
Davydenko, A., & Fildes, R. (2016). Forecast Error Measures: Critical Review and Practical Recommendations. In Business Forecasting: Practical Problems and Solutions. John Wiley & Sons
A: Why not compare $RMSE = \sqrt{MSE}$  and $MAE = MAD$? 
Actually, 
$MAE \leq RMSE \leq \sqrt{n} MAE$ for regression models:


*

*lower bound: each case contributes the same absolute amount of error $e$:
$RMSE = \sqrt{\frac{1}{n} \sum e_i^2} = \sqrt{\frac{1}{n} n e^2} = e = MAE$

*upper bound: a single case having error $e$ while all other cases have 0 error:
$MAE = \frac{e}{n}$
$RMSE = \sqrt{\frac{1}{n} \sum e_i^2} = \sqrt{\frac{1}{n} e^2} = \sqrt{\frac{1}{n} (n MAE)^2} = \sqrt{n} MAE$


($MAE \leq RMSE \leq \sqrt{MAE}$ for classification with partial class memberships  $y_i$ and/or $\hat y_i$  are $\in [0, 1]$ -- i.e. they can actually take values in between 0 and 1).


*

*upper bound: here, $e_i$ is $\leq 1$, so
$MAE = \frac{n_{wrong}}{n}$
$RMSE = \sqrt{\frac{1}{n} \sum e_i^2} = \sqrt{\frac{1}{n} n_{wrong}} = \sqrt{MAE}$
(This upper bound occurs for integer $n_{wrong}$, if you go for partial/fractional class membership and thus also for $e_i \in [0, 1]$, things get a bit more complicated because you need to take into account that the maximum possible error can be less than 1, and you may have a "leftover" $e_i < 1$ which both lower the upper bound a bit further.)


If the RMSE is close the MAE, you have many small deviations, if it is close to its upper bound, there are few grossly wrong predictions.
A: To decide which point forecast error measure to use, we need to take a step back. Note that we don't know the future outcome perfectly, nor will we ever. So the future outcome follows a probability distribution. Some forecasting methods explicitly output such a full distribution, and some don't - but it is always there, if only implicitly.
Now, we want to have a good error measure for a point forecast. Such a point forecast $F_t$ is our attempt to summarize what we know about the future distribution (i.e., the predictive distribution) at time $t$ using a single number, a so-called functional of the future density. The error measure then is a way to assess the quality of this single number summary.
So you should choose an error measure that rewards "good" one number summaries of (unknown, possibly forecasted, but possibly only implicit) future densities.
The challenge is that different error measures are minimized by different functionals. The expected MSE is minimized by the expected value of the future distribution. The expected MAD is minimized by the median of the future distribution. Thus, if you calibrate your forecasts to minimize the MAE, your point forecast will be the future median, not the future expected value, and your forecasts will be biased if your future distribution is not symmetric.
This is most relevant for count data, which are typically skewed. In extreme cases (say, Poisson distributed sales with a mean below $\log 2\approx 0.69$), your MAE will be lowest for a flat zero forecast. See here or here or here for details.
I give some more information and an illustration in What are the shortcomings of the Mean Absolute Percentage Error (MAPE)? That thread considers the mape, but also other error measures, and it contains links to other related threads.

In the end, which error measure to use really depends on your Cost of Forecast Error, i.e., which kind of error is most painful. Without looking at the actual implications of forecast errors, any discussion about "better criteria" is basically meaningless.
Measures of forecast accuracy were a big topic in the forecasting community some years back, and they still pop up now and then. One very good article to look at is Hyndman & Koehler "Another look at measures of forecast accuracy" (2006).
Finally, one alternative is to calculate full predictive densities and assess these using proper scoring-rules.
