I am assuming the following model:

$Y = \beta X + \epsilon$

Here both $X$ and $Y$ are matrices. I fit the least squares model without any regularization and get the matrix $\beta$. I would like to do the following now. I want to pick some subset $S$ of rows of $\beta$ using forward stepwise selection such that these rows minimize the squared loss. How should one go about it?

  • 7
    $\begingroup$ It's probably best to avoid going about it. If that doesn't make sense / you want to understand why, it may help to read my answer here: algorithms-for-automatic-model-selection. Why are you opposed to regularization? $\endgroup$ – gung - Reinstate Monica Apr 9 '13 at 4:20
  • $\begingroup$ I am not opposed to regularization. Its just that I was not planning to use it for the case now. $\endgroup$ – Shishir Pandey Apr 11 '13 at 10:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.