Linear regression has model hyperparameters such as number of predictors. For example in a autoregressive time series model AR(p), p is the number of predictors. To find which value of p to find we have 2 approaches:
- adjusted metrics such as Bayesian Information Criterion (BIC) and others. These metrics depends on an error (such as RSS) but only on the number of parameters, i.e. the metric is penalized for high number of parameters to avoid overfitting.
- validation/cross-validation on a train/test split data.
Which one is preferable and when?
For example I am reading a book where they use BIC to find p in AR(p) and not crossvalidation, the application is forex prediction. I was wondering why.