1
$\begingroup$

So I am given the following question

Data set sample5.txt has a 20-dimensional input $x$ in $\mathbb{R}^{20}$ but we suspect that many of these are actually irrelevant. Could you model the function $y = f(x)$ while - at the same time - figuring which dimensions contribute to the output?

So it is a feature selection task - I understand that. But I'm sort of confused by the

at the same time

part. I know many feature selection algorithms but they do not actually produce models for the data, they just produce decisions regarding which features are important and which are not. Conversely, a model (alone) doesn't really give much information regarding which features are important and which are not.

Perhaps you could do simple linear regression and then select features based on the weights (but I have never heard of anyone doing this). Or do you think that I am over-analyzing the question and what I should do is simply do feature selection first, then create the model?

$\endgroup$
1
$\begingroup$

I think that if this is a homework question then you are supposed to say so?

Anyway, lots of shrinkage methods will perform feature selection for you at the same time. Start with the Lasso (http://www-stat.stanford.edu/~tibs/lasso.html), which you seem to be on the right track towards (for orthogonal features it will select features on the weights, exactly as you say).

$\endgroup$
  • $\begingroup$ definitely not a homework question. done with school a long time ago :) Thanks, I hadforgotten about lasso. Some of the features are clearly uncorrelated with the target variable, so its quite clear which are unimportant, it was just "at the same time" thing that threw me. Thanks again! $\endgroup$ – user1893354 Sep 19 '13 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.