In the book Introduction to statistical learning(Section 6.1) it is mentioned that

Forward stepwise selection can be applied even in the high-dimensional setting where n(instances) < p(predictors).

(A least square model to be fit)

But is it not when number of predictors in the incremental selection procedure just increases beyond n, the model thus fit gives infinitely many solutions?

And I won't be able to use this procedure of subset selection beyond n=p.

So my question is , why are the authors telling that forward selection is feasible when p > n?


You cannot select $p>n$ in stepwise selection, as you correctly note. But you may start the selection path from some $p_0<n$ and it might suffice to select $p^*<n$ for successful prediction. As long as you start with $p_0<n$ and do not allow $p$ to go over $n$, the procedure works.

It is in this sense that forward selection is feasible when $p>n$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.