Match model selection strategies with modelling objectives

Question

I am confused trying to match different model selection strategies with different modelling objectives. (Unfortunately, my confusion is reflected in the length of the post. Please be patient.)

Model selection strategies:
1. Pick the model with the best-behaved residuals (i.e. residuals that appear the closest to $i.i.d.$, that is, have the lowest autocorrelations, least evidence of heteroskedasticity, etc.)
2. Pick the model with the lowest BIC
3. Pick the model with the lowest AIC

Modelling objectives:
A. Descriptive: Find patterns in the data and model them so that no patterns remain in the model residuals (perhaps a poor definition?).
B. Explanatory: minimize model bias as defined in Hastie et al. "Elements of Statistical Learning" (2009) p. 223, equation (7.9). Loosely speaking, find the model that would best approximate the true data generating process if we were able to estimate the model perfectly.
C. Predictive: minimize the expected prediction error for a new observation as in the reference above. That is in contrast to B because we have to account for imperfect estimation due to the limited sample size at hand.

Regarding B and C, see also Shmueli "To Explain or To Predict" (2011). Once again, to contrast B and C, note that under B there is no penalty on model complexity since we do not care about estimation precision; model bias is the only thing that matters. Meanwhile, for C both model bias and model estimation variance matter.

---

Based on intuition (and to some extent on Rob J. Hyndman's blog post "To Explain or To Predict" -- but I don't blame him for my own mistakes), I would guess that strategy 1. matches with objective A, 2. with B and 3. with C. (Honestly, my definition of A was partly based on 1., which may lead to biased inference.) However, I am far from certain... E.g. the match between 2. and B does not make sense as BIC penalizes model complexity while in B we should ignore it. But then again BIC will asymptotically select the true model from the candidate model set, if the true model is in the set.

Question: How do model selection strategies 1., 2., 3. match/mistacth modelling objectives A, B, C?

---

Here is a concrete example. Suppose I have a time series $\{x_t\}$ (a sample from 1 to T) and two models, AR(1) and AR(2), indexed as model 1 and model 2. Suppose further that BIC(1) < BIC(2) suggesting model 1 is "better". Suppose also that the residuals of model 2 are well behaved while the residuals of model 2 have statistically significant partial autocorrelation (PACF) at lag 2.

It appears that residual diagnostics are in conflict with BIC with respect to model selection. But perhaps there is no actual conflict and the two actually match different modelling goals? For example, model selection based on residual diagnostics matches the goal of descriptive modelling while BIC-based selection matches the goal of explanatory modelling?

---

Somewhat related posts are this and this.

Answer 1

I´m going to attempt an answer because I think I can bring some clarity to the questions: the answer might not match, in that case please ignore.

First AIC and BIC. These measures are in a way very similar. When fitting a model, adding parameters can reduce the error. Adding parameters can also lead to overfitting: this is the famous bias and variance trade-off. If not familiar, look it up. AIC and BIC are both measures that correct the error measure of a model for the number of parameters, preventing you from adding to many parameters and overfitting your data. One of the two is more conservative (I forgot which one.)

For the residuals. Some models give optimal parameter estimations if assumptions are met; for example normal i.i.d. and homoskedastic residuals. Not meeting the assumptions does not mean that the model is of no value, merely that the estimations might be off or that error estimates are not similar across observations. Looking at the residuals it might be possible to add a parameter that for example removes heteroskedasticity in the model. This would improve the model.

Putting it together very loosely: AIC and BIC are there to protect you from variance. Residual assumptions can give directions on how to assess or improve a model in a more qualitative way (some times removing bias).

These measures have to be balanced and are not necessarily mutually prevailing.