Under certain conditions, AIC is an efficient model selection criterion. I understand this roughly as if AIC will tend to select the model that will yield the largest expected likelihood of a new data point from the same data generating process or population (among all models that we are selecting from). This makes AIC the preferred choice if the goal is prediction and the evaluation of predictions is the likelihood.
However, we do not always evaluate prediction accuracy by the likelihood. There are other means of evaluating predictions such as, say, mean squared error (MSE) or mean absolute error (MAE). Questions:
- Is AIC still the model selection method of choice if prediction accuracy is evaluated by these loss functions (MSE, MAE)?
- What could be a good counterexample, preferably among the well-known loss functions? I.e. what loss function would not favor AIC as the model selection criterion?
- How can we characterize the entirety of loss functions for evaluating prediction accuracy that are compatible with AIC being the method of choice for model selection?