Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then control the FDR to select the significant feature variables, what's the disadvantage of this comparing to the lasso or group lasso algorithm, which choose the feature variables simultaneously?

Basically the question could be reduced to what's the difference between regressing one response variable on multiple covariates vs on 1 covariate at a time?

share|improve this question
I agree with gung. Also note that model discussion has been discussed a lot lately here, that might help you further. As for your approach, one problem are correlated predictors or predictors that are only influential after controlling for others. – Erik Aug 28 '12 at 14:30
1  
(+1) @gung for the gentle approach. Do note that of the 9 questions, only 4 have received answers (and two others have been closed). I still suspect you are correct in that the OP is simply unfamiliar with this aspect of the interface. – cardinal Aug 28 '12 at 14:30
2  
I see two very different questions here. Your second question is handled in any competent text on linear regression theory, but it is not the same as you first question. – cardinal Aug 28 '12 at 14:34
1  
@gung Thanks. I modified the questions. and I also choose the correct answers for the previous questions. I am sorry that I did not do that since I do not know that. – Honglang Wang Aug 28 '12 at 15:55
The main difference is your multiple t-testing/FDR approach fails to consider the significance of a feature conditional on other features being in the model. LASSO (via the LARS/forward stagewise algorithm) is adding a little bit of each predictor until another predictor is equally correlated with the residuals. The solution path implicitly considers all features when determining the model. – muratoa Aug 30 '12 at 17:52

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.