This is one of the dirty little secrets of statistics: quite often, a model is fitted and/or selected based on one criterion, e.g., likelihood maximization, and then evaluated on a different metric, which may or may not be related to the first one.
For instance, I work in forecasting. Most papers fit models using a variant of likelihood maximization, but then rank different models (specifically: their forecasts) based on the mape or the mae.
This does not make sense.
Yes, optimizing on one KPI may be easier than optimizing on the one we will use to evaluate. For instance, likelihoods are typically nice, smooth and infinitely differentiable, and the gradients and Hessians may have nice closed forms and be easy to evaluate, whereas the MAPE or MAE are not even differentiable everywhere. However:
- This reminds me of the old story of the drunk that lost his keys and was searching for them under a street light - not because that was where he lost them, but because the light was better there.
- I almost never see a discussion of this trade-off. Very few people seem to be aware of the fact that their software uses a particular criterion to fit a model, and that this may be different from the one they later use to evaluate it.
However, one point where this does not apply is when you maximize a likelihood to fit a predictive distribution, then summarize this distribution with a single number, which is a functional of the distribution chosen to minimize a specified loss function. This is a Good Thing, and it is valid because it explicitly separates the prediction from the decision aspect.