Stepwise model selection by AIC

Question

I am learning about performing stepwise model selection by AIC and having some questions:

1. What is the regularization parameter for step-AIC?
2. In what way is forward step-AIC an evolution of univariate screening? What is the potential problem regarding the computational burden?

Answer 1

Preamble: Avoid doing stepwise model selection via AIC if there are plan to use the model for anything else other than prediction. Please see this thread for more details: Algorithms for automatic model selection. To your side questions:

1. The "regularisation parameter" is number of parameters in the model.
2. It really isn't an evolution; more like a slightly more principled version of it. It is a bit unclear what you mean by "computational burden" but realistically, the actual computational burden is not huge, especially within the context of GLMs where we can warm-start our IRWLS.

Again, avoid using stepwise regression unless there is an extremely good reason (e.g. expensive to collect new data for multiple variables, very constrained memory spaces, etc.) and there are not intention to interpret the model causality/statistically/whatever. CV.SE has some excellent threads on the matter, see for starters: A more definitive discussion of variable selection, Choosing variables to include in a multiple linear regression model and Stepwise AIC - Does there exist controversy surrounding this topic?.