1
$\begingroup$

I am trying to verify if these two terms "Feature selection" and "Feature subset selection" relate to the same term. After viewing a number of articles it seems that both are used interchangeably, while I was initially assuming that the latter relates to ensemble feature selection where not a single feature subset but a number of feature subsets is calculated.

Can you confirm that these two are just different names for the same task or is there some significant difference similar to my initial assumptions?

$\endgroup$
1
  • $\begingroup$ what is the context? $\endgroup$
    – TPArrow
    Apr 11, 2017 at 8:21

1 Answer 1

2
$\begingroup$

I like "An Introduction to Statistical Learning with Applications in R" by G.James et al., so I took a look at this book.

According to the book, it seems like feature selection is more wide term that includes methods of subset selection. P. 218:

"In this chapter, we see some approaches for automatically performing feature selection or variable selection — that is, for excluding irrelevant variables from a multiple regression model.

<...>

Subset Selection. This approach involves identifying a subset of the p predictors that we believe to be related to the response. We then fit a model using least squares on the reduced set of variables.

Shrinkage. This approach involves fitting a model involving all p predictors.However, the estimated coefficients are shrunken towards zero relative to the least squares estimates. This shrinkage (also known as regularization) has the effect of reducing variance. Depending on what type of shrinkage is performed, some of the coefficients may be estimated to be exactly zero. Hence, shrinkage methods can also perform variable selection".

So it seems that for feature selection we may use subset selection (i.e. best subset selection, backward or forward selection) or shrinkage methods such as lasso regression.

Hope this helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.