# Stepwise AIC - Does there exist controversy surrounding this topic?

I've read countless posts on this site that are incredibly against the use of stepwise selection of variables using any sort of criterion whether it be p-values based, AIC, BIC, etc.

I understand why these procedures are in general, quite poor for the selection of variables. gung's probably famous post here clearly illustrates why; ultimately we are verifying a hypothesis on the same dataset we used to come up with the hypothesis, which is just data dredging. Furthermore, p-values are affected by quantities such as collinearity and outliers, which heavily skew results, etc.

However, I've been studying time series forecasting quite a bit lately and have come across Hyndman's well respected textbook in which he mentions here the use of stepwise selection to find the optimal order of ARIMA models in particular. In fact, in the forecast package in R the well known algorithm known as auto.arima by default uses stepwise selection (with AIC, not p-values). He also criticizes p-value based feature selection which aligns well with multiple posts on this website.

Ultimately, we should always cross validate in some way at the end if the goal is to develop good models for forecasting/prediction. However, surely this is somewhat of a disagreement here when it comes to the procedure itself for evaluation metrics other than p-values.

Does anyone have any opinions on the use of stepwise AIC in this context, but also in general out of this context? I have been taught to believe any stepwise selection is poor, but to be honest, auto.arima(stepwise = TRUE) has been giving me better out of sample results than auto.arima(stepwise = FALSE) but perhaps this is just coincidence.

• One of the few things that forecasters can agree on is that selecting one "best" model usually works less well than combining multiple different models. – S. Kolassa - Reinstate Monica Nov 18 '18 at 15:28

• (+1) Very informative. The approach using AIC/BIC or other information criteria shouldn't be mixed with inferential statistics using $p$-values in any case according to Burnham & Anderson's book "Model selection and multimodel inference: A practical information-theoretic approach." – COOLSerdash Nov 17 '18 at 20:04