# Forward Stepwise selection

I am assuming the following model:

$Y = \beta X + \epsilon$

Here both $X$ and $Y$ are matrices. I fit the least squares model without any regularization and get the matrix $\beta$. I would like to do the following now. I want to pick some subset $S$ of rows of $\beta$ using forward stepwise selection such that these rows minimize the squared loss. How should one go about it?

• It's probably best to avoid going about it. If that doesn't make sense / you want to understand why, it may help to read my answer here: algorithms-for-automatic-model-selection. Why are you opposed to regularization? – gung - Reinstate Monica Apr 9 '13 at 4:20
• I am not opposed to regularization. Its just that I was not planning to use it for the case now. – Shishir Pandey Apr 11 '13 at 10:08